Configuring Data Retention and Data Purge Policies
J
Joe PATTON
started a topic
28 days ago
Hello All,
I have a quick question regarding data retention policies and purge jobs for Semarchy xDM data hubs. Master History(MH_) and Golden History(GH_) tables make up most of the top 20 objects in our Semarchy database environments. Our data models currently have no data retention policies. We have also grown the number of our Semarchy xDM non-prod environments in recent months. I'm just looking for ideas on how to trim the storage footprint of our non-prod environments after running a refresh from the production environment. I would probably leave our QA environment alone, but I'm thinking about setting a minimal data retention policy in our other non-production environments and scheduling purges to reduce the size of the data hub history tables. I would be interested in hearing about any other ways to slim down our lower Semarchy xDM environments after refreshing from the production environment.
Regards,
Joe Patton
Best Answer
A
Alexia SIMOND
said
17 days ago
Hello Joe,
If you need to have exactly the same set of data for this lower environment as the PRD, then indeed purge seems to be the way to go.
It would help you regain a bit of space, without compromising the master and golden data scope.
If you require more information or have further questions on this concept, please let me know.
Otherwise, I'm leaving here some documentation for reference:
Joe PATTON
Hello All,
I have a quick question regarding data retention policies and purge jobs for Semarchy xDM data hubs. Master History(MH_) and Golden History(GH_) tables make up most of the top 20 objects in our Semarchy database environments. Our data models currently have no data retention policies. We have also grown the number of our Semarchy xDM non-prod environments in recent months. I'm just looking for ideas on how to trim the storage footprint of our non-prod environments after running a refresh from the production environment. I would probably leave our QA environment alone, but I'm thinking about setting a minimal data retention policy in our other non-production environments and scheduling purges to reduce the size of the data hub history tables. I would be interested in hearing about any other ways to slim down our lower Semarchy xDM environments after refreshing from the production environment.
Regards,
Joe Patton
Hello Joe,
If you need to have exactly the same set of data for this lower environment as the PRD, then indeed purge seems to be the way to go.
It would help you regain a bit of space, without compromising the master and golden data scope.
If you require more information or have further questions on this concept, please let me know.
Otherwise, I'm leaving here some documentation for reference:
https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Design/security/data-retention.html
https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Manage/data-purges.html
https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Design/workflows/workflow-create.html#_retention_policy
Are you looking only at data location perspective? Or repository as well?
BR,
Alexia
Alexia SIMOND
Hello Joe,
If you need to have exactly the same set of data for this lower environment as the PRD, then indeed purge seems to be the way to go.
It would help you regain a bit of space, without compromising the master and golden data scope.
If you require more information or have further questions on this concept, please let me know.
Otherwise, I'm leaving here some documentation for reference:
https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Design/security/data-retention.html
https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Manage/data-purges.html
https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Design/workflows/workflow-create.html#_retention_policy
Are you looking only at data location perspective? Or repository as well?
BR,
Alexia
-
Can we reset Matches and run again on match rule change or add a new match rule?
-
"Unmerge" records
-
Turn off match rules to speed up an integration job
-
Can anyone tell me how to load a Fuzzy-Matched entity ... but skip the matching happening auto-magically?
-
Importing CSV in Fuzzy Matched Entity Does Not Trigger Consolidation
-
How can I trigger a "match on child records"?
-
How can I Configure Most Frequent Values in Survivorship Rules?
-
Deterministic or probabilistic matching
-
Machine Learning and AI for matching
-
Prevent loads from replacing values overridden by users
See all 53 topics