IBM Cloud Docs
Installing and using dbt-watsonx-presto

Installing and using dbt-watsonx-presto

This section covers the steps to install and use dbt-watsonx-presto.

Procedure

  1. Run the following command on your system to install dbt-watsonx-presto.

    pip install dbt-watsonx-presto
    
  2. Run the following command to verify the dbt version.

    dbt –version
    
  3. Run the following command to create a dbt project.

    dbt init <project_name>
    
    1. Select a Presto number and enter it. Example: for [1] presto, enter 1.
    2. If you already have a project with the same name, you must confirm whether to overwrite profiles.yml. Enter Y to confirm or N to discard.
  4. Set up the profiles.yml file. For more information, see Configuration (setting up your profile).

  5. To test the connection, run:

    cd <project_name>
    dbt debug
    
  6. Create a CSV file inside the seeds folder to seed the data into watsonx.data. For example:

    id, value
    1,100
    2,200
    3,300
    4,400
    

    You might encounter errors when executing seed files because dbt cannot handle all the data types based on the data in the connector. To resolve this, you can explicitly define the data types that dbt should use. Go to <project_name>/dbt_project.yml and add:

    seeds:
      <project_name>:
        <seed_file_name>:
          +column_types:
            <col_1>: datatype
            <col_2>: datatype
    

    For example:

    seeds:
      demo:
        sample:
          +column_types:
            value: VARCHAR(20)
    

    Column names that are specified here should match with the columns in the CSV files.

    Do not use extra spaces in CSV files for seeding. If you include extra spaces, you must use same number of spaces while querying that in the models to avoid errors.

  7. Run the seeds by using the following command to create a table and insert the data.

    cd <project_name>
    dbt run
    
  8. In <project_name>/models, you have the models that perform the operations. By default, dbt sets the operations as view. You can create the tables or views by one of the following methods:

    • Specify inside the models (applicable for that model only)

      {{ config(materialized='table/view') }}
      

      If this statement is commented out using (--), dbt still uses the configuration. To disable it, remove it entirely or comment it in Jinja style ({# … #}).

    • Specify in dbt_project.yml (applicable for all models)

      models:
        <project_name>:
          <model_folders>:
            +materialized: table/view
      

      For example:

      models:
        demo:
          example:
            +materialized: table
      

      Only select statements are supported within models.

    The semicolon (;) character is restricted in models.

  9. Run the models by using the following command to create the tables or views.

    cd <project_name>
    dbt run
    

    You can also specify the tests that you want:

    models:
      - name: <model_name>
        description: "some description"
        columns:
          - name: <col_name>
            description: "some description"
            data_tests:
              - <test_name_1>
              - <test_name_2>
    

    For example:

    models:
      - name: my_first_dbt_model
        description: "A starter dbt model"
        columns:
          - name: id
            description: "The primary key for this table"
            data_tests:
              - unique
              - not_null
    

    Connectors must support Create Table as Select (CTAS) for dbt runs to work.

  10. To generate the documents about the actions performed, run:

    cd <project_name>
    dbt docs generate
    dbt docs serve
    

    By default, it runs on localhost:8080. To change the port, run:

    dbt docs serve –-port <port_number>