IBM Cloud Docs
Integrating with IBM Manta Data Lineage

Integrating with IBM Manta Data Lineage

Integrating Manta with watsonx.data allows you to capture and publish jobs, runs, and dataset events from Spark and Presto through the Manta UI, providing full visibility into data flows and transformations.

Before you begin

  • Create a data source definition and an OpenLineage connection.
  • Create an OpenLineage metadata import in your project and run it.

For more information, see Preparing data for IBM Manta Data Lineage.

Procedure

  1. Log in to the watsonx.data console.

  2. From the navigation menu, go to Configurations > IBM Manta Data Lineage.

  3. Enter the following details:

    IBM Manta Data Lineage integration
    Field Description
    Lineage ingestion endpoint Enter the IBM Cloud host endpoint URL where the data lineage service is activated.
    API key Enter the API key. For information about generating an API key, see Creating an API key

  4. Click Save to save the details. You can edit the saved details by clicking Edit.

  5. Click Enable to enable Manta Data Lineage.

Data lineage takes effect for new jobs that are started after enabling it. Previous and ongoing jobs will not display data lineage. Currently, it supports viewing lineages for CREATE TABLE AS (CTAS) and INSERT INTO SELECT operations in Manta Data Lineage.

What to do next

  • You can view data lineage for your assets. For more information, see Viewing data lineage.
  • You can manage and adjust your lineage graph to get comprehensive visibility and control of the data pipeline. For more information, see Managing data lineage graph.