Document versioning and MVCC

Multi-version concurrency control (MVCC) is how IBM® Cloudant® for IBM Cloud® databases ensures that all of the nodes in a database's cluster contain only the newest version of a document.

Since IBM Cloudant databases are eventually consistent, it is necessary to prevent inconsistencies from arising between nodes as a result of synchronizing between outdated documents.

Multi-Version Concurrency Control (MVCC) enables concurrent read and write access to an IBM Cloudant database. MVCC is a form of optimistic concurrency. It makes both read and write operations on IBM Cloudant databases faster because the database locks on either read or write operations isn't necessary. MVCC also enables synchronization between IBM Cloudant database nodes.

Revisions

Every document in an IBM Cloudant database has a _rev field that indicates its revision number.

A revision number is added to your documents by the server when you insert or modify them. The number is included in the server response when you make changes or read a document. The _rev value is constructed by using a combination of a simple counter and a hash of the document.

The two main uses of the revision number are to help in the following cases:

Determine what documents must be replicated between servers.
Confirm that a client is trying to modify the latest version of a document.

You must specify the previous _rev when you update a document or else your request fails and returns a 409 error.

_rev must not be used to build a version control system because it is an internal value that is used by the server. Therefore, older revisions of a document are transient, and removed regularly.

However, you can query a particular revision by using its _rev, but older revisions are regularly deleted by a process called compaction. You can query a particular document revision by using its _rev in order to obtain a history of revisions to your document. However, a consequence of compaction is that you cannot rely on a successful response. If you need a version history of your documents, a solution is to create a new document for each revision.

Distributed databases and conflicts

Distributed databases work without a constant connection to the main database on IBM Cloudant, which is itself distributed, so updates based on the same previous version can still be in conflict.

To find conflicts, add the query parameter conflicts=true when you retrieve a document. The response contains a _conflicts array with all conflicting revisions.

To find conflicts for multiple documents in a database, write a view.

The following map function is an example that emits all conflicting revisions for every document that has a conflict.

See the following example of a map function to find documents with a conflict:

function (doc) {
    if (doc._conflicts) {
        emit(null, [doc._rev].concat(doc._conflicts));
    }
}

You might regularly query this view and resolve conflicts as needed, or query the view after each replication.

Steps to resolve conflicts

After you find a conflict, you can resolve it in four steps: get, merge, upload, and delete, as shown later.

Let's consider an example of how to resolve a conflict. Suppose that you have a database of products for an online shop. The first version of a document might look like the following example:

{
    "_id": "74b2be56045bed0c8c9d24b939000dbe",
    "_rev": "1-7438df87b632b312c53a08361a7c3299",
    "name": "Samsung Galaxy S4",
    "description": "",
    "price": 650
}

As the document doesn't have a description yet, someone might add one.

See the second version of the document, which is created by adding a description:

{
    "_id": "74b2be56045bed0c8c9d24b939000dbe",
    "_rev": "2-61ae00e029d4f5edd2981841243ded13",
    "name": "Samsung Galaxy S4",
    "description": "Latest smartphone from Samsung",
    "price": 650
}

At the same time, someone else - working with a replicated database - reduces the price.

See a different revision, conflicting with the previous one because of different price value:

{
    "_id": "74b2be56045bed0c8c9d24b939000dbe",
    "_rev": "2-f796915a291b37254f6df8f6f3389121",
    "name": "Samsung Galaxy S4",
    "description": "",
    "price": 600
}

The two databases are then replicated. The difference in document versions results in a conflict.

Get conflicting revisions

You identify documents with conflicts by using the conflicts=true option.

See the following example of finding documents with conflicts:

https://$ACCOUNT.cloudant.com/products/$_ID?conflicts=true

See the following example response that shows conflicting revisions that affect documents:

{
    "_id":"74b2be56045bed0c8c9d24b939000dbe",
    "_rev":"2-f796915a291b37254f6df8f6f3389121",
    "name":"Samsung Galaxy S4",
    "description":"",
    "price":600,
    "_conflicts":["2-61ae00e029d4f5edd2981841243ded13"]
}

The version with the changed price was chosen arbitrarily as the latest version of the document. The conflict with another version is noted by providing the ID of that other version in the _conflicts array. In most cases, this array has only one element, but many conflicting revisions might exist.

Merge the changes

To compare the revisions to see what changed, your application gets all of the versions from the database.

See the following example commands to retrieve all versions of a document from the database:

https://$ACCOUNT.cloudant.com/products/$_ID
https://$ACCOUNT.cloudant.com/products/$_ID?rev=2-61ae00e029d4f5edd2981841243ded13
https://$ACCOUNT.cloudant.com/products/$_ID?rev=1-7438df87b632b312c53a08361a7c3299

Since the conflicting changes are for different fields of the document, it is easy to merge them.

For more complex conflicts, other resolution strategies might be required:

Time based - use the first or last edit.
User intervention - report conflicts to users and let them decide on the best resolution.
Sophisticated algorithms - for example, 3-way merges of text fields.

For a practical example of how to implement a merge of changes, see this project with sample code.

Upload the new revision

The next step is to create a document that resolves the conflicts, and update the database with it.

See the following example document that merges changes from the two conflicting revisions:

{
    "_id": "74b2be56045bed0c8c9d24b939000dbe",
    "_rev": "3-daaecd7213301a1ad5493186d6916755",
    "name": "Samsung Galaxy S4",
    "description": "Latest smartphone from Samsung",
    "price": 600
}

Delete old revisions

Finally, you delete the old revisions by sending a DELETE request to the URLs with the revision you want to delete.

See the following example request to delete an old document revision by using HTTP:

DELETE https://$ACCOUNT.cloudant.com/products/$_ID?rev=2-61ae00e029d4f5edd2981841243ded13

See the following example request to delete an old document revision:

curl -H "Authorization: Bearer $API_BEARER_TOKEN" -X DELETE "$SERVICE_URL/events/0007241142412418284?rev=2-9a0d1cd9f40472509e9aac6461837367"

import com.ibm.cloud.cloudant.v1.Cloudant;
import com.ibm.cloud.cloudant.v1.model.DeleteDocumentOptions;
import com.ibm.cloud.cloudant.v1.model.DocumentResult;

Cloudant service = Cloudant.newInstance();

DeleteDocumentOptions documentOptions =
    new DeleteDocumentOptions.Builder()
        .db("events")
        .docId("0007241142412418284")
        .rev("2-9a0d1cd9f40472509e9aac6461837367")
        .build();

DocumentResult response =
    service.deleteDocument(documentOptions).execute()
        .getResult();

System.out.println(response);

const { CloudantV1 } = require('@ibm-cloud/cloudant');
const service = CloudantV1.newInstance({});

service.deleteDocument({
  db: 'events',
  docId: '0007241142412418284',
  rev: '2-9a0d1cd9f40472509e9aac6461837367'
}).then(response => {
  console.log(response.result);
});

from ibmcloudant.cloudant_v1 import CloudantV1

service = CloudantV1.new_instance()

response = service.delete_document(
  db='events',
  doc_id='0007241142412418284',
  rev='2-9a0d1cd9f40472509e9aac6461837367'
).get_result()

print(response)

deleteDocumentOptions := service.NewDeleteDocumentOptions(
  "events",
  "0007241142412418284",
)
deleteDocumentOptions.SetRev("2-9a0d1cd9f40472509e9aac6461837367")

documentResult, response, err := service.DeleteDocument(deleteDocumentOptions)
if err != nil {
  panic(err)
}

b, _ := json.MarshalIndent(documentResult, "", "  ")
fmt.Println(string(b))

The previous Go example requires the following import block:

import (
   "encoding/json"
   "fmt"
   "github.com/IBM/cloudant-go-sdk/cloudantv1"
)

All Go examples require the service object to be initialized. For more information, see the API documentation's Authentication section for examples.

Now, conflicts affecting the document are resolved. You can verify the status by running GET to the document again with the conflicts parameter set to true.