Any architecture changes in SDP for Data Certification
V
Vinoth KUMAR C
started a topic
4 minutes ago
Hi All,
I am reaching out to see if there are any architectural changes made to data certification process in SDP.
In xDM, every record part of the load goes through each of the certification tasks before published to GD tables, so we will have to wait till the load completes to see any record part of the load to be available for consumption. Load is considered as a single unit rather than each of the records in the load treated independently with multiple threads executing them separately to fasten the process and data availability.
For a given entity parallelization of loads also little tricky if I were to run multiple loads one through batch file and another through APIs for the same entity, both these are running as sequential loads and one has to wait for another. I understand submitting to multiple queues for the same entity could cause data inconsistencies. This one of the major challenges in customer having integrations through multiple channels.
Would be really helpful if you could share some light on this whether it is still a limitation in SDP too. Appreciate your inputs. Thanks!
Vinoth KUMAR C
Hi All,
I am reaching out to see if there are any architectural changes made to data certification process in SDP.
In xDM, every record part of the load goes through each of the certification tasks before published to GD tables, so we will have to wait till the load completes to see any record part of the load to be available for consumption. Load is considered as a single unit rather than each of the records in the load treated independently with multiple threads executing them separately to fasten the process and data availability.
For a given entity parallelization of loads also little tricky if I were to run multiple loads one through batch file and another through APIs for the same entity, both these are running as sequential loads and one has to wait for another.
I understand submitting to multiple queues for the same entity could cause data inconsistencies. This one of the major challenges in customer having integrations through multiple channels.
Would be really helpful if you could share some light on this whether it is still a limitation in SDP too.
Appreciate your inputs. Thanks!