Start a new topic

How to improve duplicate review processing time in xDM

Hi All,


Even though we don't have large match groups in our system, the usual review and merge process where stewards need to click the suggestion and the duplicate manager populates the details takes time.


Am looking for options to improve the duplicate review processing time in xDM and would really appreciate your inputs and suggestions from your experience.

Exporting the suggestions and reviewing them offline and run it through custom SQL load is one of the options am thinking, but it has its own issues and involves 2-3 extra steps. Thanks!


Hello Vinoth,


For those "performances" topics we usually advise to activate below loggers to capture the query ran on the DB:

<Root level="TRACE">

<Logger level="TRACE" name="org.springframework.jdbc.core.JdbcTemplate"/>

<Logger level="TRACE" name="org.springframework.jdbc.core.StatementCreatorUtils"/>


Careful, they are very verbose, so we usually advise to:

  • Do this test on a low environment with little activity.
  • Prepare as much as possible the use case in the UI.
  • Clear the PDE.
  • Activate the loggers.
  • Reproduce the use case.
  • Once done, quickly revert the logging configuration to default.
  • Extract the pde.
  • And if the file begins with the mention "This is a continuation of log file .../work/Catalina/localhost/semarchy/eclipse/workspace/.metadata/.bak_xxx.log", then extract all the content of this metadata folder (because the files get rewritten with the content of each new entry in the pde).


Once you have the logs, extract the query in question.

You may need to replace the parameters (?) by their values that are stated just below the query.


And once you have a runnable query, then execute it on DB side with an explain plan to check where the there might be some possible improvement.

It could help you determine if maybe new indexes are needed, or maybe that statistics needs to be recomputed.

Or even that a specific join is taking time because it may be based on non-optimized attributes...

Many things could be at play here.


Of course, if you have a specific case you'd like to report us, please feel free to open a ticket for it.


Thank you

HI Alexia,


Thanks for the details, we will check and get back incase of additional help.

Hi Vinoth,


Of course, good luck.

Login to post a comment