LDIO HTTP In Poller

Ldio:HttpInPoller

The LDIO Http In Poller is a basic Http Poller that will poll a target URL on a specified interval.

Config

General properties

Property Description Required Default Example Supported values
url Target URL to poll from. Yes N/A http://example.com/my-api HTTP and HTTPS urls (lists are supported)
cron Cron expression to declare when the polling should take place 1 Yes2 N/A */10 * * * * * Spring Cron Expression
interval Polling interval declared in ISO 8601 format. Yes2 N/A PT1S ISO 8601 formatted String
continueOnFail Indicated if continue if polling results in failure No true true true or false

LDIO Http Requester properties

Authentication properties

Property Description Required Default Supported values Example
auth.type The type of authentication required by the LDES server No NO_AUTH NO_AUTH, API_KEY or OAUTH2_CLIENT_CREDENTIALS OAUTH2_CLIENT_CREDENTIALS
auth.api-key The api key when using auth.type ‘API_KEY’ No N/A String myKey
auth.api-key-header The header for the api key when using auth.type ‘API_KEY’ No X-API-KEY String X-API-KEY
auth.client-id The client identifier when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’ No N/A String myId
auth.client-secret The client secret when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’ No N/A String mySecret
auth.token-endpoint The token endpoint when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’ No N/A HTTP and HTTPS urls http://localhost:8000/token
auth.scope The Oauth2 scope when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’ No N/A HTTP and HTTPS urls http://localhost:8000/token

Retry properties

Property Description Required Default Supported values Example
retries.enabled Indicates if the http client should retry http requests when the server cannot be reached. No true Boolean value true
retries.max Max number of retries the http client should do when retries.enabled = true No 5 Integer 100
retries.statuses-to-retry Custom comma seperated list of http status codes that can trigger a retry in the http client. No N/A Comma seperated list of Integers 410,451

When retries are enabled, the following statuses are always retried, regardless of the configured statuses-to-retry:

  • 429
  • 5xx (500 and above)

Rate limit properties

Property Description Required Default Supported values Example
rate-limit.enabled Indicates if the http client should limit http requests when calling the server. No false true or false false
rate-limit.limit Limit of requests per period, which is defined below, that the http client should do when rate-limit.enabled = true No 500 Integer 100
rate-limit.period Period in which the limit of requests, which is defined above, can be reached by the http client when rate-limit.enabled = true No PT1M ISO 8601 Duration PT1H

Http headers

Property Description Required Default Supported values Example
http.headers.[].key/value A list of custom http headers can be added. A key and value has to be provided for every header. No N/A String role
Example Http Requester config 
      config:
        http:
          headers:
            - key: role
              value: developer
            - key: alt-role
              value: programmer
        auth:
          type: API_KEY
          api-key: my-secret
          api-key-header: x-api-key
        retries:
          enabled: true
          max: 10
          statuses-to-retry: 410,451
        rate-limit:
          enabled: true
          period: P1D
          limit: 1000

Multiple urls

The Http In Poller supports polling multiple endpoints. Example configuration:

name: Ldio:HttpInPoller
config:
  auth:
    type: API_KEY
    api-key: my-key
    api-key-header: X-API-Key
  url:
    - https://webhook.site/6cb49dd1-aa05-4e77-8870-f06903805b30
    - https://webhook.site/e8078b99-4b09-496d-baa8-8ba309dec6b6
  interval: PT3S

When using multiple endpoints, the other config (auth config, interval, etc.) applies to all endpoints.

Pausing

When paused, this component will stop making any of the scheduled HTTP-calls. When resumed, it will restart these calls as if the component had been restarted, meaning any configured periods will start counting from the moment the pipeline was resumed instead of when it was originally created.

  1. The cron schedules are in timezone ‘UTC’. ↩

  2. Either choose the ‘cron’ option or the ‘interval’. However, the interval property will become deprecated. ↩ ↩2