Developing custom Cloud Pak for Data connector code

Downloading the custom-crawler-docs.zip file in Discovery 2.2.1 and later

In Discovery version 2.2.1 and later, perform the following steps to download the custom-crawler-docs.zip file to your local machine. You need root access to an installed Discovery instance:

1. Log in to your Discovery cluster.

2. Enter the following command to obtain your crawler pod name:

   oc get pods | grep crawler
   

   You might see output that looks like this:

   wd-discovery-crawler-57985fc5cf-rxk89     1/1     Running     0          85m
   

3. Enter the following command to obtain the custom-crawler-docs.zip file, replacing {crawler-pod-name} with the crawler pod name that you obtained in step 2:

   oc exec {crawler-pod-name} -- ls -l /opt/ibm/wex/zing/resources/ \
   | grep custom-crawler-docs
   

   You might see output that is similar to the following:

   -rw-r--r--. 1 dadmin dadmin 59451 Jan 19 03:50 custom-crawler-docs-${build-version}.zip
   

4. Enter the following command to copy the custom-crawler-docs.zip file to the host server, replacing {build-version} with the build version number in step 3:

   oc cp {crawler-pod-name}:/opt/ibm/wex/zing/resources/custom-crawler-docs-${build-version}.zip custom-crawler-docs.zip
   

5. Enter the following command to expand the custom-crawler-docs.zip file:

   unzip custom-crawler-docs.zip -d custom-crawler-docs-primary
   

   If necessary, copy the custom-crawler-docs.zip file to the development server.

   If your local machine does not have the unzip utility, try using the gunzip command instead, or see the documentation of the operating system of your local machine for other alternatives to expand compressed files.

   If you are using a version of Discovery that is earlier than 2.1.2 and you want to access the custom-crawler-docs.zip file, enter the following command: scp root@{instance_name}:/root/bob/sdk/custom-crawler-docs.zip {local_directory}.

For information about downloading the custom-crawler-docs.zip file to Discovery 2.2.0 and earlier, see Downloading the custom-crawler-docs.zip file to Discovery 2.2.0 and earlier.

Understanding the custom-crawler-docs.zip file

The custom-crawler-docs.zip file expands into a directory named custom-crawler-docs-primary that includes the following contents:

custom-crawler-docs-primary
├── README.md
├── build.gradle
├── config
│   ├── README.md
│   ├── messages.properties
│   └── template.xml
├── scripts
│   └── manage_custom_crawler.sh
├── settings.gradle
├── src
│   └── main
│       └── java
│           └── com
│               └── ibm
│                   └── es
│                       └── ama
│                           └── custom
│                               └── crawler
│                                   └── sample
│                                       └── sftp
│                                           └── SftpCrawler.java
|                                           └── SftpSecurityHandler.java
└── wexlib
    ├── META-INF
    │   └── MANIFEST.MF
    ├── README.md
    ├── ama-zing-custom-crawler-{version_numbers}-javadoc.jar
    └── ama-zing-custom-crawler-{version_numbers}.jar

15 directories, 12 files