LDIO Http Enricher

Ldio:HttpEnricher

A transformer which allows to send a GET or POST HTTP request to a dynamic URL provided by the model. The response is converted to linked data and added to the incoming model.

Config

Property	Description	Required	Default	Supported values	Example
adapter.name	This transformer requires an ldio-adapter to convert the responses to linked data.	Yes	N/A	Paths of supported LDIO Adapters	Ldio:RdfAdapter
adapter.config.xxx	Optional config that may be required by the adapter	No	N/A	Paths of supported LDIO Adapters	Ldio:RdfAdapter
url-property-path	Path defining the url that needs to be selected on the model for the http request.	Yes	N/A	Valid property paths	http://example.org/url
header-property-path	Path defining the headers that needs to be selected on the model for the http request.	No	N/A	Valid property paths	http://example.org/header
body-property-path	Path defining the body that needs to be selected on the model to be added when a POST http request is used.	No	N/A	Valid property paths	http://example.org/meta/http://example.org/body
http-method-property-path	Path defining the http method that needs to be selected on the model for the http request.	No	GET	GET or POST	GET

LDIO Http Requester properties

Authentication properties

Property	Description	Required	Default	Supported values	Example
auth.type	The type of authentication required by the LDES server	No	NO_AUTH	NO_AUTH, API_KEY or OAUTH2_CLIENT_CREDENTIALS	OAUTH2_CLIENT_CREDENTIALS
auth.api-key	The api key when using auth.type ‘API_KEY’	No	N/A	String	myKey
auth.api-key-header	The header for the api key when using auth.type ‘API_KEY’	No	X-API-KEY	String	X-API-KEY
auth.client-id	The client identifier when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’	No	N/A	String	myId
auth.client-secret	The client secret when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’	No	N/A	String	mySecret
auth.token-endpoint	The token endpoint when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’	No	N/A	HTTP and HTTPS urls	http://localhost:8000/token
auth.scope	The Oauth2 scope when using auth.type ‘OAUTH2_CLIENT_CREDENTIALS’	No	N/A	HTTP and HTTPS urls	http://localhost:8000/token

Retry properties

Property	Description	Required	Default	Supported values	Example
retries.enabled	Indicates if the http client should retry http requests when the server cannot be reached.	No	true	Boolean value	true
retries.max	Max number of retries the http client should do when retries.enabled = true	No	5	Integer	100
retries.statuses-to-retry	Custom comma seperated list of http status codes that can trigger a retry in the http client.	No	N/A	Comma seperated list of Integers	410,451

When retries are enabled, the following statuses are always retried, regardless of the configured statuses-to-retry:

• 429
• 5xx (500 and above)

Rate limit properties

Property	Description	Required	Default	Supported values	Example
rate-limit.enabled	Indicates if the http client should limit http requests when calling the server.	No	false	true or false	false
rate-limit.limit	Limit of requests per period, which is defined below, that the http client should do when rate-limit.enabled = true	No	500	Integer	100
rate-limit.period	Period in which the limit of requests, which is defined above, can be reached by the http client when rate-limit.enabled = true	No	PT1M	ISO 8601 Duration	PT1H

Http headers

Property	Description	Required	Default	Supported values	Example
http.headers.[].key/value	A list of custom http headers can be added. A key and value has to be provided for every header.	No	N/A	String	role

Example Http Requester config

      config:
        http:
          headers:
            - key: role
              value: developer
            - key: alt-role
              value: programmer
        auth:
          type: API_KEY
          api-key: my-secret
          api-key-header: x-api-key
        retries:
          enabled: true
          max: 10
          statuses-to-retry: 410,451
        rate-limit:
          enabled: true
          period: P1D
          limit: 1000

Note that all adapters are supported. When the adapter requires additional config, this can be added as seen below in the example.

Example:

This example contains a JsonToLdAdapter which needs “core-context” as config. At the bottom there is also “auth” config provided for the request executor.

        - name: Ldio:HttpEnricher
          config:
            adapter:
              name: Ldio:JsonToLdAdapter
              config:
                core-context: file:///ldio/jsonld/observation.jsonld
            url-property-path: <http://example.org/url>
            header-property-path: <http://example.org/meta>/<http://example.org/headers>
            body-property-path: <http://example.org/meta>/<http://example.org/body>
            http-method-property-path: <http://example.org/meta>/<http://example.org/method>
            auth:
              type: API_KEY
              api-key: my-secret
              api-key-header: x-api-key