How to run

We advise running the LDES Server as a Docker Image which we provide via Docker Hub:

• Latest Official version:
• Latest Alpha version:

To decide which version to take, visit the Release Management Advice and visit the LDES Server Github Release Page for an overview of all the releases.

LDES Server Config

The LDES Server provides a variety of tweaking options to configure it to your ideal use case.

An example basic config can be found here:

ldes-server.yml:

springdoc:
  swagger-ui:
    path: /v1/swagger
ldes-server:
  host-name: "http://localhost:8080"
spring:
  datasource:
    url: jdbc:postgresql://localhost:5432/test
    username: admin
    password: admin
  batch:
    jdbc:
      initialize-schema: always
      
rest:
  max-age: 120
  max-age-immutable: 604800

Here is an explanation provided for all the possibilities on how to tweak and configure your LDES Server:

Feature
Property	Description	Required	Default value
API endpoints documentation
springdoc.api-docs.path	The url1 that will point to the Open API documentation. More Open-API documentation	No
springdoc.swagger-ui.path	The url1 that will point to the Swagger documentation. More Swagger UI documentation	No	true
URL Configuration
server.address	This is the url that will be used by the server application to bind to. This is especially useful when exposing actuator endpoints publicly. However, it must be known if the address cannot be found or is unavailable, the application will be unable to start.	No
ldes-server.host-name	This is the url that will be used throughout the fragment names. This should therefor point to a resolvable url.	Yes
ldes-server.use-relative-url	Determines if the resources hosted on the server are constructed with a relative URI. For more	No	false
Ingest/Fetch
rest.max-age	Time in seconds that a mutable fragment can be considered up-to-date	No	60
rest.max-age-immutable	Time in seconds that an immutable fragment should not be refreshed	No	604800
PostgreSQL Storage2
spring.datasource.url	URL that points to the PostgreSQL server
spring.datasource.username	Username to log into provided PostgreSQL instance
spring.datasource.password	Password to log into provided PostgreSQL instance
Bucketisation & pagination batching
ldes-server.fragmentation-cron	Defines how often Fragmentation Service will check for unprocessed members (when present, trigger fragmentation job).	No	*/30 * * * * *
Fragment Compaction
ldes-server.compaction-cron	Defines how often the Compaction Service will check the fragments3	No	0 0 0 * * *
ldes-server.compaction-duration	Defines how long the redundant compacted fragments will remain on the server	No	PD7
ldes-server.deletion-cron	Defines how often the redundant compacted fragments will be checked for deletion3	No	0 0 0 * * *
Retention (Retention Policies)
ldes-server.retention-cron	Defines how often the Retention Service will check the members3	No	0 0 0 * * *
Ports
ldes-server.ingest.port	Defines on which port the ingest endpoint is available	No	8080
ldes-server.fetch.port	Defines on which port the fetch endpoints are available	No	8080
ldes-server.admin.port	Defines on which port the admin endpoints are available4	No	8080

Note 1: The specified url will be prefixed by an optional server.servlet.context-path

Note 2: Since the 3.0 release, the MongoDB got replaced with a PostgreSQL implementation. For Migration instructions, please check the below migration paragraph.

Note 3: Unix usually supports a cron expression of 5 parameters, which excludes seconds. However, the spring annotation @Scheduled adds a 6th parameter to support seconds. The cron schedules are in timezone ‘UTC’.

More information about this can be found in the spring documentation

Note 4: When using the swagger API with separate port bindings, the swagger API will always be available under the admin port.

Docker Compose

services:
  ldes-server:
    container_name: basic_ldes-server
    image: ldes/ldes-server
    environment:
      - SPRING_CONFIG_LOCATION=/config/
    volumes:
      - ./ldes-server.yml:/config/application.yml:ro
    ports:
      - 8080:8080
    networks:
      - ldes
    depends_on:
      - ldes-postgres
  ldes-postgres:
    container_name: ldes-postgres
    image: postgres:14-alpine
    ports:
      - 5432:5432
    environment:
      - POSTGRES_PASSWORD=admin
      - POSTGRES_USER=admin
      - POSTGRES_DB=test
    networks:
      - ldes
networks:
  ldes:
    name: quick_start_network

Migration to 3.0

Since the Mongodb implementation got replaced with a PostgreSQL one, a migration path has been provided. This migration path is only needed for those who wish to move their 2.x LDES servers to the latest version.

To enable this, please first take in the 3.0.0 release and add the following properties to your config:

ldes-server:
  migrate-mongo: true
spring:
  data:
    mongodb:
      uri: # connection string pointing to mongodb instance
  batch:
    jdbc:
      initialize-schema: always

ALTERNATIVE MIGRATION: Besides the provided migration solution, It is also possible to set up your new 3.x Server next to your 2.x Server and then use and LDES workbench to send over the data from the 2.x Server to the 3.x Server. This pipeline in LDIO should contain an LDES Client to read data from the 2.x Server and an LDIO Http Out to send data towards the 3.x Server.