Uploading a file

Uploading a file

1. Click the upload icon in the input box on Conversations or go to Data and metrics > Data sources > Uploaded files.

2. On Data and Metrics, select the project that you want to upload the files to.

3. Browse for the files on your local drive, and select one or more files to upload. You can also drop files into the drop zone.

   Progress messages appear at different stages of the upload:

   • Preparing - In this stage, watsonx BI prepares the data in the file for metadata enrichment.

   • Enriching - The data in the file then undergoes metadata enrichment, which adds business context metadata to your data to prepare it for conversations.

   • Ready - Metadata enrichment completes successfully.

4. Click the Review link to review the enrichment results.

   Metadata enrichment adds business context to your data in the form of business terms, labels, and descriptions. Check to make sure that these additions are accurate and meaningful.

   Any changes that you make to the enrichment results save automatically.

   If the uploaded file has more than 10,000 rows, click Review > Columns > Select a column. Check the Statistics section in the Details panel to ensure that data sampling captured the distinct values. If certain columns are fewer than expected, click Edit enrichment and change Sample size to Percentage. Enter 10 in the Percent of the data set field and enter 10000 in the Maximum number of rows field. Click Enrich all assets to re-run enrichment.

5. (Optional) Click Ask a question to ask questions about the data in the uploaded file.

When a file successfully uploads to a project, it is listed as a data asset in the project view.

You can use uploaded files as data sources in the create metrics flow and also define relationships between them for more complex, multi-step BI queries that might require data from two or more flat files. For more information, see Data modeling watsonx BI.

Managing uploaded files

Replacing an uploaded file

You can update the contents of an uploaded file by adding a file with the same name and format to the project. IBM watsonx BI prompts you to specify whether you want to overwrite the existing file or create a new asset. In both cases, the data in the file undergoes metadata enrichment.

Deleting an uploaded file

To delete an uploaded file, click the Delete option from the menu next to the uploaded file name.

Deleting a file removes it from the project permanently. This action might impact semantic data models that were created with references to the deleted file.

Supported file types

The upload file feature in watsonx BI supports the following flat files:

Country,City
Canada,Ottawa
United Kingdom,London 

Supported data formats

Each column in the uploaded file is scanned to determine whether it represents a character, numeric (integers, decimal, or double), temporal (date, timestamp, or time), or a Boolean data type. The file upload feature supports each of these data types.

Supported column data formats include:

• Date data types

  • YYYY-MM-DD

  • YYYY/MM/DD

  • DD-MM-YYYY

  • DD/MM/YYYY

  • MM-DD-YYYY

  • MM/DD/YYY

• Timestamp data types

  • YYYY-MM-DD HH:mm:ss

  • YYYY-MM-DD HH:mm:ss.SS

  • YYYY-MM-DDTHH:mm:ss

  • YYYY-MM-DDTHH:mm:ss.SS

    Fractional seconds (ss in the pattern) can contain up to nine digits.

• Boolean data type with 'true' or 'false' values

• Values with just digits that represent an integer, big integer, double, or decimal type

• Empty values are treated as null type

Values representing dates and timestamps need to be in the following format:

• Values that represent a month must be between 1-12

• Values that represent a day must be a valid day in the associated month.

• Values representing an hour must be between 1-24.

• Values representing a minute must be between 0-59.

• Time values that use up to 12 hours can be designated with AM (before noon) and PM (after noon)

• Column names must be unique

Troubleshooting file upload errors

You might run into errors during the file upload process. Click the View details link under the error message for more details.

Some errors can be avoided by revising the file and uploading again. For example, file upload can fail if the file is blank or contains a blank worksheet. File enrichment might fail if it contains a value that doesn't match the data type in the rest of the file, an unsupported data type, or if the file contains one or more rows that have a different number of columns than expected.

Other errors might require you to contact IBM support for further assistance.