Portal

Summarized Solution

On a high level this installation needs to setup a RDS cluster where the database will be hoisted and a setup of xDM docker image on ECS container. To access both DB and xDM, the respective inbound rules are needed to open the connection ports.

Detailed Solution

References

https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Install/aws/deploy-aws-instance.html

https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Install/database-configuration.html

Prerequisites and requirements

Review the information in this section before you begin the installation.

You will need the following to install Semarchy xDM on AWS:

1. An AWS subscription. If you don’t have an AWS subscription, create a free account before you begin. This user requires administrator privileges.
2. The AWS command-line interface (CLI). It is installed and configured with your AWS account. For more information, see Getting started with the AWS CLI.
3. One AWS Virtual Private Cloud (VPC). It is configured with at least two subnets from different availability zones. For more information, see Create an IPv4-enabled VPC and subnets using the AWS CLI.
4. A SQL management tool, such as DBeaver, pgAdmin or another one.

Step 1. Check the AWS credentials

To access AWS services with the AWS CLI, you need at minimum an AWS account and IAM credentials. 

Installing or updating to the latest version of the AWS CLI

https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html

Create an access key and secret token to configure AWS.

This is the only time that the secret access key can be viewed or downloaded. You cannot recover it later. However, you can create a new access key any time.

To check all iam users – use the following command:

aws iam list-users

aws iam get-user

aws rds create-db-cluster \
    --db-cluster-identifier devpoc-aurora-cluster \
    --engine aurora-postgresql \
    --master-username dev_postgres_user\
    --master-user-password dev_postgres \
    --database-name dev_postgres \
    --tags Key=xdm-ecs

aws rds create-db-instance \
    --db-cluster-identifier devpoc-aurora-cluster \
    --db-instance-identifier dev-postgres-instance \
    --db-instance-class db.t4g.medium \
    --engine aurora-postgresql \
    --db-subnet-group-name default \
    --tags Key=xdm-ecs

psql "host=your-db-instance-endpoint port=5432 dbname=your-database user=your-username sslmode=verify-full sslrootcert=/path/to/rds-combined-ca-bundle.pem"

Step 3. Create an inbound rule for DB instance accessibility

1. Go to the DB instance on RDS

2. Go to Connectivity & Security and under Security section click on link for VPC security groups.

3. Go to Security group ID

4. Add an inbound rule for port 5432 

Step 4. Prepare Database for installation

Using any DB client try to connect to DB. You should be able to successfully connect and can only see public schema. Under users ensure to see the master user you've used in the DB creation command. In our case it is dev_postgres_user.

You can get connect details as below.

For Host name and post go to the DB instance RDS - Databases - <your cluster> - <your database instance> - Connectivity and Security

 

Go to Configuration for DB Name

Go back to your SQL management tool and execute the following scripts. Ensure to provide appropriate DB names, users names if you are using different than what are used in below scripts. Refer to below documentation too for more details.

https://www.semarchy.com/doc/semarchy-xdm/xdm/latest/Install/database-configuration.html

CREATE SCHEMA extensions;
GRANT USAGE ON SCHEMA extensions TO PUBLIC;
ALTER DEFAULT PRIVILEGES IN SCHEMA extensions GRANT EXECUTE ON FUNCTIONS TO PUBLIC;
ALTER DATABASE dev_postgres SET SEARCH_PATH TO "$user",public,extensions;
 
CREATE EXTENSION IF NOT EXISTS "uuid-ossp"     with schema extensions;
CREATE EXTENSION IF NOT EXISTS "fuzzystrmatch" with schema extensions;

-- Create the repository user and schema
CREATE USER semarchy_repository WITH PASSWORD 'semarchy_repository';

-- Use the following syntax for PostgreSQL 9
-- CREATE USER <repository_user> WITH UNENCRYPTED PASSWORD '<repository_password>';


-- The following command is required only for PostgreSQL running on Amazon RDS.
-- It grants access to the repository to the RDS superuser.
GRANT semarchy_repository TO dev_postgres_user;


CREATE SCHEMA semarchy_repository AUTHORIZATION semarchy_repository;
-- Create the repository read-only user
CREATE USER semarchy_repository_ro WITH PASSWORD 'semarchy_repository_ro';

-- Use the following syntax for PostgreSQL 9
-- CREATE USER <repository_readonly_user> WITH UNENCRYPTED PASSWORD '<repository_readonly_password>';

GRANT CONNECT ON DATABASE dev_postgres to semarchy_repository_ro;

-- Set the search path to include the repository
ALTER ROLE semarchy_repository_ro SET SEARCH_PATH TO "$user", semarchy_repository,public,extensions;

-- Run the following commands after the repository creation
-- Grant select privileges on the profiling tables
GRANT USAGE ON SCHEMA semarchy_repository TO semarchy_repository_ro;

CREATE USER devdloc WITH PASSWORD 'devdloc';
-- Use the following syntax for PostgreSQL 9
-- CREATE USER <data_location_user_name> WITH UNENCRYPTED PASSWORD '<data_location_user_password>';

-- The following command is required only for PostgreSQL running on Azure or Amazon RDS.
-- It grants access to the data location to the Azure or RDS superuser.
GRANT devdloc TO dev_postgres_user;

CREATE SCHEMA devdloc AUTHORIZATION devdloc;
-- Grant data-loading privileges to the data location owner
GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA semarchy_repository TO devdloc;

Refresh the DB and you should be able to see all the schemas and users created using above scripts.

Step 5. Deploy xDM on ECS

1. Go to Amazon Elastic Containers Service - Clusters. 

2. Click on Create cluster. Add cluster name and Default namespace.

3. Keep other configuration as is or change as per your requirements. Click on Create. 

It will take few minutes to create the cluster. Wait until it is successfully created.

4. Now we have to create a service. Click on cluster which is just created and then go to services section. Click on Create.

Under environment section keep the configurations as is.

5. In the Deployment Configuration section, click on Task Definition.

6. Click on create a new task definition.

7. Provide a task definition name and in Infrastructure requirements add the configuration as per you requirements e.g Launch Type, Operating System, CPU and Memory.

8. To provide Image URI in Conatainer-1 section, search the docker image as per your requirements. Follow below steps.

• Go to docker hub and search semarchy/xdm
• In semarchy/xdm, click on Tags and search for the xdm version that you want to install. Copy the image URL. For e.g semarchy/xdm:2024.1.8

9. Go back to task definition and in container-1 section add the details as shown below.

10. Add resource allocation limit as per your requirements.

11. Add the environment variables. Verify the DB environment variable from your DB cluster and instance.

 - XDM_REPOSITORY_DRIVER=org.postgresql.Driver
 - XDM_REPOSITORY_URL= jdbc:postgresql://dev-postgres-instance.cmyokm9dd1is.eu-north-1.rds.amazonaws.com:5432/dev_postgres
 - XDM_REPOSITORY_USERNAME=semarchy_repository
 - XDM_REPOSITORY_PASSWORD=semarchy_repository

# Repository read-only datasource credentials
- XDM_REPOSITORY_READONLY_USERNAME=semarchy_repository_ro
- XDM_REPOSITORY_READONLY_PASSWORD=semarchy_repository_ro

# Setup Token
- SEMARCHY_SETUP_TOKEN=mySecretValue

# (Optional) Customizable context path
- CONTEXT_PATH=/semarchy

11. Keep the other configuration as is or update as per your requirements. Click on Create.

12. Go back to service creation screen it must be already opened in your browser. You will see the task under Family.

13. In the Networking section, ensure that the Public IP is turned ON.

14. Keep other configuration as it is or update as per your requirements and click on Create.

The deployment will take some time.

Once the deployment is complete, you will see a service created.

Step 5. Create an inbound rule to open port 8080 

1. Click on service, go to task and click on Task

2. Under networking, click on security group

3. Add and inbound rule to open port 8080.

Step 6. Access the xDM application

1. Go back to networking and note down the public ip.

xDM will be accessible using URL http://16.16.219.198:8080/semarchy/login.do

2. Click on the link, the application will ask for secret token. Add the value as provided in the environment variable. 

3. Accept the policies.

4. Provide the admin password to access the application and click on install.

5. Login to xDM using admin credentials

6. To configure the data location, click on Configuration - Datasources and click on + icon.

7. Add the required details and test.

xDM on AWS ECS is ready for the development.