Meroxa

The Meroxa Developer Hub

Welcome to the Meroxa developer hub. You'll find comprehensive guides and documentation to help you start working with Meroxa as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started    Guides

Amazon S3

Amazon S3 provides flexible object storage. Using Meroxa, you can sync data from other Resources to S3 in real-time.

Adding Resource

To add an Amazon S3 resource to your Meroxa Resource Catalog, you can run the following command:

meroxa add resource datalake --type s3 -u "s3://$AWS_ACCESS_KEY:[email protected]$AWS_REGION/$AWS_S3_BUCKET"

datalake is a human-friendly name to represent the S3 resource. Feel free to change as desired.

In the command above, replace the following variables with valid credentials from your S3 environment:

  • $AWS_ACCESS_KEY - AWS Access Key
  • $AWS_ACCESS_SECRET - AWS Access Secret
  • $AWS_REGION - AWS Region (e.g., us-east-2)
  • $AWS_S3_BUCKET - AWS S3 Bucket Name

Permissions

The following AWS Access Policy is required to be attached to the IAM user of the AWS_ACCESS_KEY provided in the Connection URL:

{
    "Statement": [
        {
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:DeleteObject",
                "s3:AbortMultipartUpload",
                "s3:ListMultipartUploadParts",
                "s3:ListBucketMultipartUploads"
            ],
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::<bucket-name>/*",
                "arn:aws:s3:::<bucket-name>"
            ]
        }
    ],
    "Version": "2012-10-17"
}

Creating Connectors

To configure S3 as a destination, see: S3 Destination Configuration

Updated about a month ago


Amazon S3


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.