What is a DataFlow?

Last Updated: Thu May 14 2020

Data flows are a way to organize multiple operations in a single transaction. Data flows organize a series of patch calls to create, update, or delete connected data - entities and API calls that are related to each other using foreign keys, response values, or parent/child relationships. Data flows are all-or-none -- if a single operation in the flow fails, all of the operations will rollback.

The table below compares and contrasts data flows with the API:

Application API (REST)

  • RESTful API

  • Single transaction per request

  • Fixed structure

  • Primary and Foreign Keys auto-managed

  • Parent can only update using Patch

Data Flow

  • Not RESTful but Data-driven

  • Multiple operations per request

  • Flexible structure

  • Caller manages Primary and Foreign Keys using tokens

  • Parent can Create, Update, or Delete

Envelopes, Patches, and Tokens

Patches -The basic unit of a dataflow is a patch. A patch performs a specific operation (create, update or delete) on a system object, and contains property information or references a token, for the operation.

Tokens - Patches can get or set variables (see tokens) for use in a subsequent patch. For example, a create account patch could set a variable for the account ID which will then be used to add contact information to the account.

Envelopes - Patches that are related are placed together in an envelope. An envelope defines the transactional scope of the operation. If any patch within the envelope fails, then all preceding patches will rollback and the operation will be terminated.

Benefits of Data Flows

Using data flows offers users the following benefits:

  • Transactional roll back using envelopes ensures no partial data transfers

  • Able to use response data from one patch in request data for subsequent patches

  • A single format for an integration process

Data Flow or Data Import?

Considerations to make where deciding between a Data Import  and a Data Flow include:

  • Do you intend to update a single object type or many?

  • Is there a parent-child type relationship?

  • Are you performing a series of linked/related operations?

Use Cases

Common scenarios where you might want to use data flows include

  • Import customer data and orders from CRM/CPQ

  • Migrating customer data from a legacy billing system

  • Move/Add/Changes for devices and services