Sizing the "Active" Semarchy xDM Service in Kubernetes : Portal

Start a new topic

Sizing the "Active" Semarchy xDM Service in Kubernetes

Joe Patton

started a topic about 16 hours ago

Hello All,

I'm just curious about how many Semarchy xDM customers run on Servers/VM's vs. Kubernetes deployments. We have been running in Azure Kubernetes for about 2 years. We run in our own namespaces in a "shared" cluster that is managed by a separate team of engineers. The nodepool/scale set where we run Semarchy xDM is made up of 16 core/64GB RAM Azure VM's. Due to our volume of data and issues with duplicates in some of our models/hubs, we are having issues with memory pressure on our 'active' deployment during duplicates management phases of our data certification jobs.

Our "active" deployment in our QA and PROD environments is configured with deployment spec "limit" CPU/RAM values of 6 cores and 56GB. This basically takes an entire node in the AKS node pool on deployment with no further headroom on RAM( JVM -XX:MaxRAMPercentage=75.0). Without going into detail, our most problematic data hub is currently about 1.4TB in size. The largest object in this data hub is a duplicate(du_*) table with a size of 225GB.

We are are running on version 2024.1.1, but will soon be upgrading to 2025.1.11.

From a platform support perspective, I believe that we have a few options to explore:

- work with our data stewards and mdm developers to more closely investigate duplicates/match groups for our problem entities

- consider standing up a new AKS nodepool for semarchy that is constructed of 16 core 128GB VM's

- change our platform architecture from Azure Kubernetes deployments to Azure VM's

Our solution might have more to do with our data, source data quality and model design, but I just was looking for advice on when to consider increasing the total amount of RAM available on our 'active' Semarchy service.

Regards,

Joe Patton

3 Comments

Haashim Jhungeer

said about 13 hours ago

Hi Joe,

A few thoughts before you commit to a resizing project:

Check for overmatching first. A 225GB DU_* table against a 1.4TB hub is often a sign of oversized match clusters rather than a pure volume problem. A quick query grouping your MI_<entity> table by the matching attribute(s) will show if a handful of clusters are ballooning. If so, tightening the match rule (and replaying matching) typically buys back more memory/time than adding hardware.

Your 2025.1.11 upgrade should already help. Two relevant improvements land before that version:

The UPDATE_WHILE transitive closure mode - a more memory-efficient alternative for the duplicate/closure phase - became the default behavior as of 2025.1.4 (previously an opt-in JVM flag).
A 2025.1 fix reduced workflow-related heap usage.

Worth re-testing your memory profile after the upgrade before deciding anything else.

On your three options:

Investigating match/duplicate design —- do this first; it's the only option that actually shrinks the memory working set.
Bigger node pool (128GB) —- reasonable fallback if overmatching is ruled out and pressure persists post-upgrade.
Moving off AKS to VMs —- doesn't change the underlying memory need of a large JVM processing large clusters; more of an ops/architecture call than a fix for this symptom.

Thanks,
Haashim

Joe Patton

said about 12 hours ago

Thank You Haashim,

I am more on the platform side, but I investigate into this issue further. I ran a number of intermediate queries, but there is one particular match rule that seems to be a large factor on the "location" entity in the model/data hub where we are seeing a high volume of duplicates.

select

b_matchrule,

count(*)

from du_{entity}

group by b_matchrule;

DU Distribution

Match Rule	DU Rows	% of Total DU
ExactOrganizationAndAddress	344,952,066	~97.7%
ExactZoomIdAndOrganization	5,383,677	~1.5%
OwnerIsNotNull	2,438,566	~0.7%
ExactOwnerByAndAddressAndOrganization	68,858	<0.1%
ExactOwnerByAndAddressAndOrganizationAndZoomId	37	~0%

I passed these findings and some additional information on match groups to our mdm development team. Thank you for the suggestions. I agree that adding platform capacity will not address the root cause in our current situation.

Regards,

Joe Patton

Haashim Jhungeer

said about 10 hours ago

Hi Joe,

That's a very clean diagnosis where 97.7% of your DU_* rows coming from a single rule (ExactOrganizationAndAddress) is about as clear an overmatching signature as it gets.

A couple of things that might help your MDM dev team as they dig in, based on what's typically behind this pattern:

"Exact" match rules on address + organization are prone to grouping unrelated records when the underlying values are loosely standardized (e.g., inconsistent address formatting, blank/null organization values normalizing to the same string, or shared corporate addresses like headquarters/PO boxes linking many distinct locations together). Worth checking whether the attributes feeding that rule are enriched/standardized consistently before matching.
It's also worth confirming whether ExactOrganizationAndAddress is meant to be a strict identity rule or was intended as a broader candidate-grouping rule that should be combined with an additional discriminating attribute (similar to how ExactOwnerByAndAddressAndOrganization narrows things down further in your list) - often overmatching comes from a rule that was originally scoped tighter and became a catch-all as data volume grew.
Once your dev team narrows the rule, replaying matching on the location entity (rather than reprocessing the full hub) should be enough to validate the fix without a full reload.

Since you've confirmed platform capacity isn't the root cause here, I'll leave this with you and your MDM team on the rule design side. Feel free to reach back out if you want a second pair of eyes once they've proposed a rule change, or if memory pressure persists after that fix.

Thanks and Best regards,
Haashim