We are using xDI (runtimes and analytics) version 2023.1.6 and would like to set up a logging solution with Datadog.
We would like to retrieve process execution logs stored in the H2 database and accessible via Analytics.
Is there a ready-made solution available? If not, what solutions would be possible?
Thank you for your replies.
Kind regards
Best Answer
A
Audric FREY
said
6 days ago
Hello,
There is currently no built-in / ready-made Datadog connector or plugin for xDI. That said, integrating xDI with Datadog is achievable using a standard log-shipping approach, which is the option we'd recommend.
The xDI Runtime already uses Log4j2 for its technical logging (configured in log4j.xml), and it includes internal loggers dedicated to session and action lifecycle events (Session Start, Session End, Action Start, Action End). These loggers are disabled by default (level="off"), but can be enabled.
The approach works as follows:
1. Enable the relevant internal loggers (Session Start/End, Action Start/End) in log4j.xml, and route them to a dedicated log file.
2. Configure the appender's layout to output structured JSON rather than plain text. Log4j2 supports this natively, and xDI exposes useful contextual fields through its own pattern parameters (sessionId, sessionName, sessionStatus, deliveryName, actionPath, actionStatus, duration, etc.) that can be included in the JSON output.
3. Install the Datadog Agent on the server (or as a sidecar container if you're running on Kubernetes), and configure it to tail this log file.
4. Datadog automatically parses JSON-formatted logs and exposes each field as a searchable facet, so you can filter and build dashboards on sessionName, deliveryName, sessionStatus, duration, etc.
This approach has the advantage of relying entirely on standard, already-existing xDI logging mechanisms combined with a standard Datadog log-shipping pattern, so it requires no custom development or scripting, just a Log4j2 configuration change and a Datadog Agent setup.
One point to validate on your side: the level of detail exposed by these internal loggers should be checked against your monitoring requirements to confirm it covers what you need (e.g., success/failure per action, durations, statistics).
Let us know if you'd like guidance on configuring the log4j.xml file for this, or if you have specific fields/metrics in mind that you'd want to see in Datadog, so we can confirm they are available through this logging mechanism.
Best regards
1 Comment
A
Audric FREY
said
6 days ago
Answer
Hello,
There is currently no built-in / ready-made Datadog connector or plugin for xDI. That said, integrating xDI with Datadog is achievable using a standard log-shipping approach, which is the option we'd recommend.
The xDI Runtime already uses Log4j2 for its technical logging (configured in log4j.xml), and it includes internal loggers dedicated to session and action lifecycle events (Session Start, Session End, Action Start, Action End). These loggers are disabled by default (level="off"), but can be enabled.
The approach works as follows:
1. Enable the relevant internal loggers (Session Start/End, Action Start/End) in log4j.xml, and route them to a dedicated log file.
2. Configure the appender's layout to output structured JSON rather than plain text. Log4j2 supports this natively, and xDI exposes useful contextual fields through its own pattern parameters (sessionId, sessionName, sessionStatus, deliveryName, actionPath, actionStatus, duration, etc.) that can be included in the JSON output.
3. Install the Datadog Agent on the server (or as a sidecar container if you're running on Kubernetes), and configure it to tail this log file.
4. Datadog automatically parses JSON-formatted logs and exposes each field as a searchable facet, so you can filter and build dashboards on sessionName, deliveryName, sessionStatus, duration, etc.
This approach has the advantage of relying entirely on standard, already-existing xDI logging mechanisms combined with a standard Datadog log-shipping pattern, so it requires no custom development or scripting, just a Log4j2 configuration change and a Datadog Agent setup.
One point to validate on your side: the level of detail exposed by these internal loggers should be checked against your monitoring requirements to confirm it covers what you need (e.g., success/failure per action, durations, statistics).
Let us know if you'd like guidance on configuring the log4j.xml file for this, or if you have specific fields/metrics in mind that you'd want to see in Datadog, so we can confirm they are available through this logging mechanism.
dchatel
Hello,
We are using xDI (runtimes and analytics) version 2023.1.6 and would like to set up a logging solution with Datadog.
We would like to retrieve process execution logs stored in the H2 database and accessible via Analytics.
Is there a ready-made solution available? If not, what solutions would be possible?
Thank you for your replies.
Kind regards
Hello,
There is currently no built-in / ready-made Datadog connector or plugin for xDI. That said, integrating xDI with Datadog is achievable using a standard log-shipping approach, which is the option we'd recommend.
**Recommended approach: Log4j2 + Datadog Agent (log shipping)**
The xDI Runtime already uses Log4j2 for its technical logging (configured in log4j.xml), and it includes internal loggers dedicated to session and action lifecycle events (Session Start, Session End, Action Start, Action End). These loggers are disabled by default (level="off"), but can be enabled.
The approach works as follows:
1. Enable the relevant internal loggers (Session Start/End, Action Start/End) in log4j.xml, and route them to a dedicated log file.
2. Configure the appender's layout to output structured JSON rather than plain text. Log4j2 supports this natively, and xDI exposes useful contextual fields through its own pattern parameters (sessionId, sessionName, sessionStatus, deliveryName, actionPath, actionStatus, duration, etc.) that can be included in the JSON output.
3. Install the Datadog Agent on the server (or as a sidecar container if you're running on Kubernetes), and configure it to tail this log file.
4. Datadog automatically parses JSON-formatted logs and exposes each field as a searchable facet, so you can filter and build dashboards on sessionName, deliveryName, sessionStatus, duration, etc.
This approach has the advantage of relying entirely on standard, already-existing xDI logging mechanisms combined with a standard Datadog log-shipping pattern, so it requires no custom development or scripting, just a Log4j2 configuration change and a Datadog Agent setup.
One point to validate on your side: the level of detail exposed by these internal loggers should be checked against your monitoring requirements to confirm it covers what you need (e.g., success/failure per action, durations, statistics).
Let us know if you'd like guidance on configuring the log4j.xml file for this, or if you have specific fields/metrics in mind that you'd want to see in Datadog, so we can confirm they are available through this logging mechanism.
Best regards
Audric FREY
Hello,
There is currently no built-in / ready-made Datadog connector or plugin for xDI. That said, integrating xDI with Datadog is achievable using a standard log-shipping approach, which is the option we'd recommend.
**Recommended approach: Log4j2 + Datadog Agent (log shipping)**
The xDI Runtime already uses Log4j2 for its technical logging (configured in log4j.xml), and it includes internal loggers dedicated to session and action lifecycle events (Session Start, Session End, Action Start, Action End). These loggers are disabled by default (level="off"), but can be enabled.
The approach works as follows:
1. Enable the relevant internal loggers (Session Start/End, Action Start/End) in log4j.xml, and route them to a dedicated log file.
2. Configure the appender's layout to output structured JSON rather than plain text. Log4j2 supports this natively, and xDI exposes useful contextual fields through its own pattern parameters (sessionId, sessionName, sessionStatus, deliveryName, actionPath, actionStatus, duration, etc.) that can be included in the JSON output.
3. Install the Datadog Agent on the server (or as a sidecar container if you're running on Kubernetes), and configure it to tail this log file.
4. Datadog automatically parses JSON-formatted logs and exposes each field as a searchable facet, so you can filter and build dashboards on sessionName, deliveryName, sessionStatus, duration, etc.
This approach has the advantage of relying entirely on standard, already-existing xDI logging mechanisms combined with a standard Datadog log-shipping pattern, so it requires no custom development or scripting, just a Log4j2 configuration change and a Datadog Agent setup.
One point to validate on your side: the level of detail exposed by these internal loggers should be checked against your monitoring requirements to confirm it covers what you need (e.g., success/failure per action, durations, statistics).
Let us know if you'd like guidance on configuring the log4j.xml file for this, or if you have specific fields/metrics in mind that you'd want to see in Datadog, so we can confirm they are available through this logging mechanism.
Best regards
-
How to fix error "HikariPool-3 - Connection is not available, request timed out after 100005ms."
-
How to force the decimal separator by setting user.country java option?
-
All the deliveries failed
-
How to configure the scheduler for Sql Server database ? (having a specific schema name (not dbo))
-
[Error]: ClassNotFoundException
-
Error when installing xDI Runtime on Docker
-
Error when using quartz scheduler
-
Getting issue when trying to install xdi runtime on docker
-
[startdelivery] : error Caused by: javax.net.ssl.SSLHandshakeException: The server selected protocol version TLS10 is not accepted by client preferences [TLS13, TLS12]
-
How to check runtime version ?
See all 16 topics