Start a new topic
Answered

Breakdown of Semarhcy Welcome Application page

hi ,

I am looking for support with anyone who might have an idea why the Semarchy application Login/Welcome Landing page breaks down or becomes unresponsive randomly quite a lot while all the other function of that environment seems to be working fine.

Its a scenario in which we we manually enter a hyperlink i.e. application builder or platform support other then the welcome page everything seems to be working as expected but the login landing page is unresponsive.


We have tried to restart the server/clearing the cache yet it still behaves that way.


Any help would be appreciated.




Best Answer

Hello,

we had similar issues with our test environments. Just for reference, these were sandbox environments meant for training / demo usages, so they had a lot more Semarchy models & deployments than what is typical.

Just reference, these environments were running over AWS Fargate & AWS Aurora PostgreSQL Serverless 1. The fix to our issue was two-fold:

1. We missed the X-Forwarded Headers upgrade actions from 2023.1.3. -> New parameters for java in setenv.sh

  • There was an upgrade actions mentioned regarding X-Forwaded headers being ignored by default.
  • Access to our xDM servers was propagated behind AWS Application Load Balancer.  Therefore, X-Forwaded headers being ignored led to load balancers directing traffic incorrectly which led to /semarchy/welcome timing out after login request (albeit other pages worked).
  • This was fixed by adding the "-DallowXForwardedHeaders=true"-parameter for java in setenv.sh file.
  • To note! Depending on your your architecture is setup, this may require security concerns, however, for our setup having XForwarded headers on is fine.
2. Cleaning up unnecessary old model versions from Semarchy UX / backend

  • As mentioned, our environments had a lot of models & a lot of deployments. 
  • Entry to /semarchy/welcome triggered  big SELECT's to Semarchy backend mta_* tables. After pinpointing longest queries, we found that some mta_* tables had very substantial amounts of data in them, slowing queries noticeably.
  • After more investigation we pinpointed that the large data amounts were related the the high amount of unnecessary models & deployment history in the data locations. 
  • After cleaning up 20-40 unused Semarchy models and 100s of rows of old deployment history from demo data locations (as these were not required in any way), the query times to the various mta_* tables became substantially faster.
  • All in all, both the demos & environments started working much faster after the cleanups.

Bare in mind, this is quite the special case. Our demo environments were used for training etc, therefore, we had a lot of duplicate models with 100s of deployments due to people using the environment for tutorials and testing. 

We also noticed that there was some unknown issues with AWS Aurora Serverless 1, as especially before the cleanup mentioned in point 2, the long queries seemed to cause some cascading failures on data base side especially when we hadn't done the fix mentioned in section 1. Now everything is working well, but we did also migrate to AWS Serverless 2, which might be a factor in the better performance. 

Hopefully this helps at all,
Joel Vanhalakka

image



1 Comment

Answer

Hello,

we had similar issues with our test environments. Just for reference, these were sandbox environments meant for training / demo usages, so they had a lot more Semarchy models & deployments than what is typical.

Just reference, these environments were running over AWS Fargate & AWS Aurora PostgreSQL Serverless 1. The fix to our issue was two-fold:

1. We missed the X-Forwarded Headers upgrade actions from 2023.1.3. -> New parameters for java in setenv.sh

  • There was an upgrade actions mentioned regarding X-Forwaded headers being ignored by default.
  • Access to our xDM servers was propagated behind AWS Application Load Balancer.  Therefore, X-Forwaded headers being ignored led to load balancers directing traffic incorrectly which led to /semarchy/welcome timing out after login request (albeit other pages worked).
  • This was fixed by adding the "-DallowXForwardedHeaders=true"-parameter for java in setenv.sh file.
  • To note! Depending on your your architecture is setup, this may require security concerns, however, for our setup having XForwarded headers on is fine.
2. Cleaning up unnecessary old model versions from Semarchy UX / backend

  • As mentioned, our environments had a lot of models & a lot of deployments. 
  • Entry to /semarchy/welcome triggered  big SELECT's to Semarchy backend mta_* tables. After pinpointing longest queries, we found that some mta_* tables had very substantial amounts of data in them, slowing queries noticeably.
  • After more investigation we pinpointed that the large data amounts were related the the high amount of unnecessary models & deployment history in the data locations. 
  • After cleaning up 20-40 unused Semarchy models and 100s of rows of old deployment history from demo data locations (as these were not required in any way), the query times to the various mta_* tables became substantially faster.
  • All in all, both the demos & environments started working much faster after the cleanups.

Bare in mind, this is quite the special case. Our demo environments were used for training etc, therefore, we had a lot of duplicate models with 100s of deployments due to people using the environment for tutorials and testing. 

We also noticed that there was some unknown issues with AWS Aurora Serverless 1, as especially before the cleanup mentioned in point 2, the long queries seemed to cause some cascading failures on data base side especially when we hadn't done the fix mentioned in section 1. Now everything is working well, but we did also migrate to AWS Serverless 2, which might be a factor in the better performance. 

Hopefully this helps at all,
Joel Vanhalakka

image




1 person likes this
Login to post a comment