Start a new topic
Answered

Runtimes logging with datadog

Hello,

We are using xDI (runtimes and analytics) version 2023.1.6 and would like to set up a logging solution with Datadog.

We would like to retrieve process execution logs stored in the H2 database and accessible via Analytics.

Is there a ready-made solution available? If not, what solutions would be possible?

Thank you for your replies.

Kind regards


Best Answer

Hello,


There is currently no built-in / ready-made Datadog connector or plugin for xDI. That said, integrating xDI with Datadog is achievable using a standard log-shipping approach, which is the option we'd recommend.


**Recommended approach: Log4j2 + Datadog Agent (log shipping)**


The xDI Runtime already uses Log4j2 for its technical logging (configured in log4j.xml), and it includes internal loggers dedicated to session and action lifecycle events (Session Start, Session End, Action Start, Action End). These loggers are disabled by default (level="off"), but can be enabled.


The approach works as follows:


1. Enable the relevant internal loggers (Session Start/End, Action Start/End) in log4j.xml, and route them to a dedicated log file.

2. Configure the appender's layout to output structured JSON rather than plain text. Log4j2 supports this natively, and xDI exposes useful contextual fields through its own pattern parameters (sessionId, sessionName, sessionStatus, deliveryName, actionPath, actionStatus, duration, etc.) that can be included in the JSON output.

3. Install the Datadog Agent on the server (or as a sidecar container if you're running on Kubernetes), and configure it to tail this log file.

4. Datadog automatically parses JSON-formatted logs and exposes each field as a searchable facet, so you can filter and build dashboards on sessionName, deliveryName, sessionStatus, duration, etc.


This approach has the advantage of relying entirely on standard, already-existing xDI logging mechanisms combined with a standard Datadog log-shipping pattern, so it requires no custom development or scripting, just a Log4j2 configuration change and a Datadog Agent setup.


One point to validate on your side: the level of detail exposed by these internal loggers should be checked against your monitoring requirements to confirm it covers what you need (e.g., success/failure per action, durations, statistics).


Let us know if you'd like guidance on configuring the log4j.xml file for this, or if you have specific fields/metrics in mind that you'd want to see in Datadog, so we can confirm they are available through this logging mechanism.


Best regards

1 Comment

Answer

Hello,


There is currently no built-in / ready-made Datadog connector or plugin for xDI. That said, integrating xDI with Datadog is achievable using a standard log-shipping approach, which is the option we'd recommend.


**Recommended approach: Log4j2 + Datadog Agent (log shipping)**


The xDI Runtime already uses Log4j2 for its technical logging (configured in log4j.xml), and it includes internal loggers dedicated to session and action lifecycle events (Session Start, Session End, Action Start, Action End). These loggers are disabled by default (level="off"), but can be enabled.


The approach works as follows:


1. Enable the relevant internal loggers (Session Start/End, Action Start/End) in log4j.xml, and route them to a dedicated log file.

2. Configure the appender's layout to output structured JSON rather than plain text. Log4j2 supports this natively, and xDI exposes useful contextual fields through its own pattern parameters (sessionId, sessionName, sessionStatus, deliveryName, actionPath, actionStatus, duration, etc.) that can be included in the JSON output.

3. Install the Datadog Agent on the server (or as a sidecar container if you're running on Kubernetes), and configure it to tail this log file.

4. Datadog automatically parses JSON-formatted logs and exposes each field as a searchable facet, so you can filter and build dashboards on sessionName, deliveryName, sessionStatus, duration, etc.


This approach has the advantage of relying entirely on standard, already-existing xDI logging mechanisms combined with a standard Datadog log-shipping pattern, so it requires no custom development or scripting, just a Log4j2 configuration change and a Datadog Agent setup.


One point to validate on your side: the level of detail exposed by these internal loggers should be checked against your monitoring requirements to confirm it covers what you need (e.g., success/failure per action, durations, statistics).


Let us know if you'd like guidance on configuring the log4j.xml file for this, or if you have specific fields/metrics in mind that you'd want to see in Datadog, so we can confirm they are available through this logging mechanism.


Best regards

Login to post a comment