Managing tasks

curl and jq basics

To get all active tasks and format the output nicely, call your account by using curl, and pipe the output to jq.

With jq, you filter a list of documents by their field values. This filter makes it easier to get all replication documents, or the details of just one particular view indexing task. The API reference has more information about the options.

See an example of obtaining and formatting a list of active tasks:

curl "$SERVICE_URL/_active_tasks" | jq

Monitoring view builds and search indexes

View indexes are rebuilt when a design document is updated. An update to any one of the views causes all the views in the document to be rebuilt.

Search indexes are rebuilt only when their corresponding index function is changed. For each search index that is built and for each design document with views that are changed, a new task is created for each replica and each shard in a cluster.

For example, if 24 shards with three replicas each exist, and you update two search indexes, then 24 x 3 x 2 = 144 tasks run.

To find all the view indexing tasks, pipe the curl output to jq, and let it filter the documents in the array by their type field. A corresponding command works for search indexing tasks.

In each case, the results of searching for a list of indexing tasks is a list of JSON objects: one for each of the active tasks found.

See an example of finding all view indexing tasks by filtering for the indexer type:

curl -s "$SERVICE_URL/_active_tasks" | jq '.[] | select(.type=="indexer")'

See an example of finding all search indexing tasks by filtering for the search_indexer type:

curl -s "$SERVICE_URL/_active_tasks" | jq '.[] | select(.type=="search_indexer")'

See example results after you search for view indexing tasks:

{
    "total_changes": 6435,
    "started_on": 1371118332,
    "user": "username",
    "updated_on": 1371118334,
    "type": "indexer",
    "node": "dbcore@db6.meritage.cloudant.net",
    "pid": "<0.16366.6103>",
    "changes_done": 364,
    "database": "shards/40000000-7fffffff/username/database",
    "design_document": "_design/ngrams"
}

Estimating the time to complete a task

To estimate the time needed until the indexing task is complete, monitor the number of changes_done and compare this value to total_changes. For example, if changes_done advances by 250 per second, and total_changes is 1,000,000, the task is expected to take 1,000,000 / 250 = 4,000 seconds, or about 66 minutes,to complete.

Estimates of the time to complete an indexing task can't be 100% correct. The actual time to complete the task depends on the following factors:

• The time that it takes to process each document. For instance, a view might check the type of document first, and emit new index entries for only one type.
• The size of the documents.
• The current workload on the cluster.

You must assume that these factors might combine to produce considerable inaccuracy in your estimate.

See the example of extracting the changes_done field by using jq:

curl ... | jq '.[] | select(.type=="search_indexer") | .changes_done'

Monitoring replication

To find all replication tasks, pipe the curl output to jq, and filter the documents in the array by their type field.

Make it easier to select the information about a replication process from the list of active tasks by following these steps:

1. Start the replication process by creating a document in the _replicator database.
2. Set its _id field to a known value.

See an example of finding all replication tasks, by filtering for the replication type:

curl -s "$SERVICE_URL/_active_tasks" | jq '.[] | select(.type=="replication")'

See an example of finding a specific replication task, by filtering for a known document identity:

curl ... | jq '.[] | select(.doc_id=="ID")'

See an example of finding a specific replication task, by filtering for a known replication_id:

curl ... | jq '.[] | select(.replication_id=="ID")'

See an example result after you search for a replication task:

{
    "started_on": 1371094220,
    "source_seq": "62960-sakdjflksdfjsdlkafjalskdfjlsakfjlasdkjksald",
    "source": "",
    "revisions_checked": 12,
    "continuous": true,
    "doc_id": null,
    "doc_write_failures": 0,
    "docs_read": 12,
    "target": "",
    "type": "replication",
    "updated_on": 1371118477,
    "user": "username",
    "checkpointed_source_seq": "61764-dskfjalsfjsalkfjssadjfhasdfkjhsdkfhsdkf",
    "changes_pending": 1196,
    "pid": "<0.9955.4120>",
    "node": "dbcore@db7.meritage.cloudant.net",
    "docs_written": 12,
    "missing_revisions_found": 12,
    "replication_id": "asfksdlfkjsadkfjsdalkfjas+continuous+create_target"
}

Troubleshooting stuck tasks