Developer Workflow

This document recommends a developer workflow for building integration using Doohickey Developer Tools and Doohickey Cloud Services. Doohickey is meant to be adapted to your process, but if you're not sure where to start, this is a good approach.

Workflow Considerations

• What OIH instance do you use?
• What do your developer endpoint environments look like?
• Are you building custom components or services with the templates?

Workflow Basics

The recommended developer workflow encompasses three steps:

• Flow Template Development and Development Testing
• Staging Deployment via Doohickey for Formal Testing
• Production Deployment via Doohickey

Template Development & Development Testing

The object during this phase is to create (or modify) a developer tested set of integration templates that deliver on the requirements. At the end of this phase the integration (package of templates) should be able to hand off to a testing team who interacts with the integration as an end user would. Quick feedback loops are key in this phase, so the developer can quickly see if the changes he/she has made work as expected.

Environment Set Up

During development, you'll need to frequently test your changes by using your template to deploy runnable OIH flows. You'll then have to run those flows.

There are three appropriate environment set ups to consider. Both have advantages, depending on your team's needs:

1. Deploying to a hosted shared "development" environment
2. Deploying to a hosted shared "stage" environment (the same environment is also used for staging)
3. Deploying to a locally run OIH environment

Options 1 and 2 relieve the integration developer the burden of maintaining and running a local instance of Open Integration Hub. If they aren't also building and testing components and services, this is likely a valuable time-saver. You should additionally consider moving from a shared enviroment for development and staging (option 2) to separate environments (option 1) as your team and your number of integrations scales.

Option 3 affords the most flexibility, fastest deployment feedback, and removes the possibility of someone else changing data.

You do not need to set up an OIH environment in Doohickey Cloud for development testing. We recommeend using the CLI to publish flows from your local templates directly to OIH.

Relevant CLI Commands

The following are commonly used CLI commands for this phase:

• doohickey init to set up your development project and/or intiailize your doohickey.json to an existing one.
• doohickey login <alias> --oih to log in directly to an OIH environment, without using Doohickey Cloud
• doohickey flow deploy -C <path to config example> to use a local template to generate a flow in the logged in OIH environment.
• doohickey secret create to create secrets used when development testing flow templates.

Tips & Tricks

As a developer you should try to replicate and develop for the types of configuration scenarios that the integration is expected to see in the wild. You will likely build one or more example configuration json files to mock various scenarios, used for both manual development testing and automated testing.

Unless building a basic template, it's helpful to build templates one or a few steps at a time. Consider building iteratively, until you've completed the entire end to end flow.

Consider the resources and traffic in your development or staging OIH environment and how it differs from production. You may also have more replicas of components and services in production. This should impact your implementation, but you want to make sure your environment is representative of real-world flow executions.

As much as possible, build and test logic in your IDE, before pushing to OIH. The Doohickey IDE Plugins will help you to build and test JSONata files locally and their use is highly recommended.

Staging Deployment

The objective during this phase is to deploy working integration flows using the templates the way they will be deployed in production via Doohickey Cloud. Ideally this deployment to Doohickey happens via your CI/CD process to limit possibilities for developers to make mistakes during deployment and to enable automated testing.

Environment Set Up

It's recommended that you deploy a dedicated "staging" OIH environment that roughly mimicks what your production OIH environment looks like. To create a production-like environment, consider compute resources dedicated to components and services, how many replicas of each are deployed, and how much general traffic moves through flows. It may be cost prohibitive for your staging environment to exactly replicate your production environment, but replicating multiple (even if fewer) instances of services and components is important for testing.

You should also set up an Environment for OIH staging in your Doohickey Cloud account and use Doohickey to deploy flows to that OIH environment. Except if doubling your staging environment for development testing, you should not deploy directly to staging OIH using the CLI.

Relevant CLI Commands

The following are commonly used CLI commands for this phase:

doohickey template deploy to deploy a flow template to Doohickey Cloud. Doohickey Cloud will automatically update any flows derived from the template, if applicable. Ideally, this command is run via your CI/CD process. doohickey environment create to register your OIH environment with Doohickey Cloud. Note: You will need your service URLs ready.

Tips & Tricks

As you set up your staging environment for testing the integration and its flow templates, you may also need to set up one or more tenants using doohickey tenant create. This is likely a one-time activity, but you'll do it to set up representative end user scenarios for testing.

Production Deployment

In this phase you deploy the integration template to Doohickey Cloud in a way that it can be used to create production integration flows in the production OIH. It's recommended that this step of deployment occur via your CI/CD process, specifically as a release.

Environment Set Up

You should use a dedicated production environment for all production deployments. That environment should be set up to mantain multitenancy for end customers. It should also be maintained under a higher security scrutiny to limit exposure to customer data or customer executions.

Relevant CLI Commands

The commands for deploying to production are the same as staing.

Tips & Tricks

Limit who has the ability to push updates to production. Idealy this should be enforced in your source control repository and/or CI/CD tool.