Skip to main content

PostgreSQL Amazon RDS Setup

This guide goes through the steps you need to take to make your PostgreSQL database on Amazon RDS accessible to the Meroxa Platform.

Required Steps

  1. Configure Security Groups
  2. Configure Network ACLs (if database is within a VPC)
  3. Enable Logical Replication (if using as a Source with Logical Replication CDC)
  4. Create a Read Replica (if using as a source with polling setup)

Configure Security Groups

To allow access to your PostgreSQL database from Meroxa IPs, follow these steps:

  1. Go to Amazon RDS in your AWS account.
  2. Go to Databases in the Amazon RDS menu.
  3. Click into the PostgreSQL database you'd like to use with Meroxa.
  4. In the Connectivity & security section, under Security and VPC security groups, click your security group by clicking the blue hyperlink.
  5. You should now see a list of Security Groups, click into the selected security group by clicking the blue hyperlink under Security group ID.
  6. Under Inbound rules, click the Edit inbound rules button.
  7. Click the Add rule button.
  8. Set Protocol to TCP.
  9. Set Port to the port of your database. For PostgreSQL it's 5432 by default.
  10. Set the Source to include the Meroxa IPs.

Configure Network ACLs

You only need to update this if your database is within a VPC.

To allow access to your PostgreSQL database from Meroxa IPs, follow these steps:

  1. Go to Amazon RDS in your AWS account.
  2. Go to Databases in the Amazon RDS menu.
  3. Click into the PostgreSQL database you'd like to create a replica of.
  4. In the Connectivity & security section, under Security and VPC security groups, click your security group by clicking the blue hyperlink.
  5. You should now see a list of Your VPCs. Click into the selected VPC by clicking the blue hyperlink under VPC ID.
  6. In your VPC Details, click the blue hyperlink under Main network ACL.
  7. You should now see a list of Network ACLs. Click into the selected Network ACL by clicking the blue hyperlink under Network ACL ID.
  8. You will update both the Inbound and Outbound Rules of the Network ACL to contain ALL - 0.0.0.0/0 - ALLOW.
  9. If your Inbound and Outbound Rules do not contain ALL - 0.0.0.0/0 - ALLOW—update the source to allow Meroxa IPs both Inbound and Outbound.

Enable Logical Replication

With logical replication enabled, Meroxa can pull records from PostgreSQL by reading the Write-Ahead Log (WAL), which produces change events for row-level INSERT, UPDATE, and DELETE operations and emits those change events into a stream.

Enabling logical replication will require a restart of your database.

Using an existing parameter group

If you would like to enable Logical Replication using an existing parameter group, follow these steps:

Go to Amazon RDS in your AWS account. Go to Databases in the Amazon RDS menu. Click into the PostgreSQL database you'd like to use with Meroxa. In the Configuration section, under Instance and click the blue hyperlink under Parameter group. Click the Edit parameters button. Search and set the following key/values in your DB parameter group rds.logical_replication - 1 Click the Save changes button.

Using a new parameter group

If you would like to enable Logical Replication using a new parameter group, follow these steps:

  1. Go to Amazon RDS in your AWS account.
  2. Go to Parameter groups in the Amazon RDS menu.
  3. Click the Create parameter group button.
  4. Set Parameter group family to the PostgreSQL version of your database (e.g., postgres14).
  5. Set Type to DB Parameter Group.
  6. Set Group name to an identifiable name.
  7. Click the Create button.
  8. You should now see a list of Parameter groups. Click into the parameter group you just created by clicking the blue hyperlink under Name.
  9. Click the Edit parameters button.
  10. Search and set the following key/values in your DB parameter group rds.logical_replication - 1.
  11. Click the Save changes button.
  12. Go back to Databases in the Amazon RDS menu.
  13. Click into the PostgreSQL database you'd like to use with Meroxa.
  14. Click the Modify button.
  15. Under the Additional configuration dropdown, look for DB parameter group and select your new parameter group.
  16. Click the Continue button.
  17. Under the Scheduling of modifications dialog, select Apply immediately.
  18. Click the Modify DB Instance button to confirm changes.
  19. Next, you must reboot the PostgreSQL server and can do that by clicking on the Actions menu and clicking Reboot.
  20. You will be asked to confirm the reboot. Click the Confirm button.
  21. Logical Replication should now be enabled on your PostgreSQL database.

Create a Read Replica

You can create a read-only replica to allow READ only access to a PostgreSQL database. Keep in mind, Logical Replication is not supported using this method.

  1. Go to Amazon RDS in your AWS account.
  2. Go to Databases in the Amazon RDS menu.
  3. Click into the PostgreSQL database you'd like to use with Meroxa.
  4. Click on the Actions menu and select Create read replica.
  5. Give your read replica a name and select Yes for Publicly accessible. All other defaluts can be changed if desired.
  6. Click the Create instance button.
  7. Finally, give the necessary access to the Read Replica.

Once you have properly taken the steps above to prepare your Amazon RDS setup for PostgreSQL, you can now configure your PostgreSQL resource on the platform.