IBM Cloud Docs
Querying data by using Lucene

Querying data by using Lucene

IBM Cloud Logs log data can be searched using Lucene or DataPrime. For information on querying with DataPrime, see Querying data by using DataPrime.

Searching can be used in conjunction with filtering and limiting by time.

Accessing Explore

You can query logs from the Explore screen:

  1. Launch the IBM Cloud Logs UI.

  2. Click the Explore logs icon Explore logs icon > Logs.

  3. (Optional) If you want to explore archived data from the Analyze and alert or Store and search pipelines, click Archive. To switch back to data currently maintained in the Priority insights pipeline click Priority Logs.

Using Lucene

  1. Use the < > Lucene/< > DataPrime switch to change the search type to Lucene.

  2. In the search field specify your search in Lucene format.

  3. Click the arrow at the end of the field run your query.

Lucene queries

A Lucene query is composed of Terms and Operators. Terms are extracted from the log data by the analyzer. There are two types of terms:

Single terms
This is a word in a text field.
Phrase
This is a group of words enclosed in double quotation marks (").

Types of searches

IBM Cloud Logs provides the following search types:

Free text search

A free text search is used to match terms in any field in your log data.

Field search

A field search restricts the search to the specific field that must match your search term.

Range search

A range search lets you query a range of matching numeric values.

Regex search

Regex searches let you use regular expressions to match patterns within your log data. Lucene standard operators are supported.

Regex patterns to be matched need to be enclosed in forward slashes (/).

Whenever possible use regex searches against keywords since keywords are not passed through the analyzer.

Example searches

The following are examples of different types of searches and their results.

Example free text searches
Query Results
a very interesting log message Matches logs containing these terms. They can appear in any field and in any order.
“a very interesting log message” Matches this exact phrase in any field.
Example field searches
Query Results
msg:interesting Matches logs containing this term in the msg field.
msg:“a very interesting log message!” Matches the exact phrase in the msg field.
msg.keyword:”a very interesting message!” Matches logs that contain the phrase (including the !).
Example range searches
Query Results
status_code.numeric:[200 TO 299] Matches status codes between 200 and 299 (including 200 and 299).
status_code.numeric:{199 TO 300} Matches status codes between 200 and 299 (excluding 199 and 300).
status_code.numeric:[200 TO 300} Matches status codes between 200 and 299 (including 200 but excluding 300).
status_code.numeric:{199 TO 299] Matches status codes between 200 and 299 (excluding 199 but including 299).
Example regex searches
Query Results
msg.keyword:/.*what an interesting message!.*/ Matches logs that contain the pattern “what an interesting message!” (including the !).
version.keyword:/.*v.[1-5].[0-9]{2}.*/ Matches logs that contain patterns like “v.1.24” or “v.5.69” in the version field.

Types of operators

You can use the operators AND, OR, and NOT to combine multiple filters. You can also use parenthesis to specify operator precedence when you have multiple operators in a query.

Example queries using operators
Query Results
msg:”failed transaction” AND level: “ERROR” NOT env:”staging” Matches ERROR level logs that contain the phrase “failed transaction” but not from the “staging environment”.
(msg:”failed transaction” AND (cluster:”eu” OR cluster:”us”)) NOT env:”staging” Matches logs from the “eu” or “us” clusters that contain the phrase “failed transaction” but not from the “staging environment”.

Resetting filters and search queries

You can clear all filters and queries by clicking Reset.

Saving a view

Once you have your view with the log records you want to see by using filters or searches, you can save the view for future use.

To save a view:

  1. Click Unsaved View.

  2. Enter the name for your view.

  3. Optionally indicate if you want to save the query and filters as part of the view.

  4. Optionally indicate if you want the view set as your default view when opening IBM Cloud Logs

  5. Select if you want the view to be private to you or shared with other users.

  6. Click SAVE to save your view.