Confluent Integration Guide
Step 1: Navigate to “Integrations” from the left navigation
On the Bicycle platform, locate the left sidebar and click on the "Integrations" option. This will take you to the integrations page where you can manage various connections.
Step 2: Navigate to “Available” and search for “Confluent”, click “Connect”
Step 3: Enter Bootstrap Server Details
Go to your Cluster Settings in Confluent
Find the value for "Bootstrap server" and copy
- Create new set of credentials for Bicycle
- In the “API Keys” section of the “Cluster Overview” create a new API key
- Enter the API Key and Secret in Bicycle UI
Step 4: Schema Registry (Optional)
Schema Registry URL
- From your Confluent Environment screen in the Confluent portal, locate the schema registry details.
- Copy the schema registry details
Schema Registry Credentials
- Just below the Schema Registry URL, click on “View and Manage” credentials link
- Create a new API key and secret
- Enter the API Key and secret for schema registry in Bicycle’s configuration
Data format
- Select what the format of your data is under “Deserializer”. The supported protocols are
- Probuf
- Avro
- Json
Step 5: Subscription and Testing your connection
- Let Bicycle know what all topics to consider using a regexp. Default is .* - meaning all available topics are considered
- Preview Topic: To test your connection, provide the name of the topic under ‘Preview Topic’ section
- Click on “Fetch Streams”, and you should see all your topics in the list
- Select one of the topics to view sample data for them
- Save the connection
- Now your connection is ready to be used
Walkthrough video