Event ingestion
Purpose​
The Event Ingestion REST API ingests business process related event data from any third-party system to Camunda Optimize. These events can then be correlated into an event-based process in Optimize to get business insights into business processes that are not yet fully modeled nor automated using the Camunda Platform.
Functionality​
The Event Ingestion REST API has the following functionality:
- Ingest new event data in batches, see the example on ingesting three cloud events.
- Reingest/override previously ingested events, see the example on reingesting cloud events.
CloudEvents compliance​
To provide the best interoperability possible, the Optimize Event Ingestion REST API implements the CloudEvents Version 1.0 specification, which is hosted by the Cloud Native Computing Foundation (CNCF).
In particular, the Optimize Event Ingestion REST API is a CloudEvents consumer implemented as an HTTP Web Hook, as defined by the CloudEvents HTTP 1.1 Web Hooks for Event Delivery - Version 1.0 specification. Following the Structured Content Mode of the HTTP Protocol Binding for CloudEvents - Version 1.0, event context attributes and event data is encoded in the JSON Batch Format of the CloudEvents JSON Event Format Version 1.0.
Authorization​
As required by the CloudEvents HTTP 1.1 Web Hooks for Event Delivery - Version 1.0 specification, every Event Ingestion REST API request needs to include an authorization token either as an Authorization request header, or as a URI Query Parameter named access_token.
Details on how to configure and pass this token can be found here.
Method and HTTP target resource​
POST /api/ingestion/event/batch
Request headers​
The following request headers have to be provided with every ingest request:
| Header | Constraints | Value |
|---|---|---|
| Authorization | REQUIRED | See Authorization |
| Content-Length | REQUIRED | Size in bytes of the entity-body, also see Content-Length. |
| Content-Type | REQUIRED | Must be one of: application/cloudevents-batch+json or application/json |
Request body​
JSON Batch Format compliant JSON Array of CloudEvent JSON Objects:
| Name | Type | Constraints | Description |
|---|---|---|---|
| specversion | String | REQUIRED | The version of the CloudEvents specification, which the event uses, must be 1.0. See CloudEvents - Version 1.0 - specversion. |
| id | String | REQUIRED | Uniquely identifies an event, see CloudEvents - Version 1.0 - id. |
| source | String | REQUIRED | Identifies the context in which an event happened, see CloudEvents - Version 1.0 - source. A use-case could be if you have conflicting types across different sources. For example, a type:OrderProcessed originating from both order-service and shipping-service. In this case, the source field provides means to clearly separate between the origins of a particular event. Note: The triplet of type, source, and group will be used as a unique identifier for classes of events. |
| type | String | REQUIRED | This attribute contains a value describing the type of event related to the originating occurrence, see CloudEvents - Version 1.0 - type. Note: The triplet of type, source, and group will be used as a unique identifier for classes of events. The value camunda cannot be used for this field. |
| time | Timestamp | OPTIONAL | Timestamp of when the occurrence happened, see CloudEvents - Version 1.0 - time. String encoding: RFC 3339. If not present, a default value of the time the event was received will be created. |
| data | Object | OPTIONAL | Event payload data that is part of the event, see CloudEvents - Version 1.0 - Event Data. This CloudEvents Consumer API only accepts data encoded as application/json, the optional attribute CloudEvents - Version 1.0 - datacontenttype is thus not required to be provided by the producer. Furthermore, there are no schema restrictions on the data attribute and thus the attribute CloudEvents - Version 1.0 - dataschema is also not required to be provided. Producer may provide any valid JSON object, but only simple properties of that object will get converted to variables of a process instances of an event-based process instance later on. |
| group | String | OPTIONAL | This is an OPTIONAL CloudEvents Extension Context Attribute that is specific to this API. A group identifier that may allow to easier identify a group of related events for a user at the stage of mapping events to a process model. An example could be a domain of events that are most likely related to each other; for example, billing. When this field is provided, it will be used to allow adding events that belong to a group to the mapping table. Optimize handles groups case-sensitively. Note: The triplet of type, source, and group will be used as a unique identifier for classes of events. |
| traceid | String | REQUIRED | This is a REQUIRED CloudEvents Extension Context Attribute that is specific to this API. A traceid is a correlation key that relates multiple events to a single business transaction or process instance in BPMN terms. Events with the same traceid will get correlated into one process instance of an Event Based Process. |
The following is an example of a valid propertie's data value. Each of those properties would be available as a variable in any event-based process where an event containing this as data was mapped:
{
"reviewSuccessful": true,
"amount": 10.5,
"customerId": "lovelyCustomer1"
}
Nested objects, such as customer in this example, would not be available as a variable in event-based processes where an event containing this as data value was mapped:
{
"customer": {
"firstName":"John",
"lasTName":"Doe"
}
}
Result​
This method returns no content.
Response codes​
Possible HTTP response status codes:
| Code | Description |
|---|---|
| 204 | Request successful |
| 400 | Returned if some of the properties in the request body are invalid or missing. |
| 401 | Secret incorrect or missing in HTTP Header X-Optimize-API-Secret. See Authorization on how to authenticate. |
| 403 | The Event Based Process feature is not enabled. |
| 429 | The maximum number of requests that can be serviced at any time has been reached. The response will include a Retry-After HTTP header specifying the recommended number of seconds before the request should be retried. See Configuration for information on how to configure this limit. |
| 500 | Some error occurred while processing the ingested event, best check the Optimize log. |
Example​
Ingest cloud events​
Request​
POST /api/ingestion/event/batch
Request Body:
[
{
"specversion": "1.0",
"id": "1edc4160-74e5-4ffc-af59-2d281cf5aca341",
"source": "order-service",
"type": "orderCreated",
"time": "2020-01-01T10:00:00.000Z",
"traceid": "id1",
"group": "shop",
"data": {
"numberField": 1,
"stringField": "example"
}
},
{
"specversion": "1.0",
"id": "1edc4160-74e5-4ffc-af59-2d281cf5aca342",
"source": "order-service",
"type": "orderValidated",
"time": "2020-01-01T10:00:10.000Z",
"traceid": "id1",
"group": "shop",
"data": {
"numberField": 1,
"stringField": "example"
}
},
{
"specversion": "1.0",
"id": "1edc4160-74e5-4ffc-af59-2d281cf5aca343",
"source": "shipping-service",
"type": "packageShipped",
"traceid": "id1",
"group": "shop",
"time": "2020-01-01T10:00:20.000Z"
}
]
Response​
Status 204.
Reingest cloud events​
The API allows you to update any previously ingested cloud event by ingesting an event using the same event id.
The following request would update the first cloud event that got ingested in the ingest three cloud events sample. Note that on an update, the cloud event needs to be provided as a whole; it's not possible to perform partial updates through this API.
In this example, an additional field newField is added to the data block of the cloud event with the id 1edc4160-74e5-4ffc-af59-2d281cf5aca341.
Request​
POST /api/ingestion/event/batch
Request body:
[
{
"specversion": "1.0",
"id": "1edc4160-74e5-4ffc-af59-2d281cf5aca341",
"source": "order-service",
"type": "orderCreated",
"time": "2020-01-01T10:00:00.000Z",
"traceid": "id1",
"group": "shop",
"data": {
"numberField": 1,
"stringField": "example",
"newField": "allNew"
}
}
]
Response​
Status 204.