De-emphasize RDS Proxy in RDI setup documentation

content/operate/rc/databases/rdi/rds-proxy.md

@@ -0,0 +1,112 @@
+---
+Title: RDS Proxy setup for RDI
+alwaysopen: false
+categories:
+- docs
+- operate
+- rc
+description: Set up RDS Proxy for Redis Data Integration (not recommended).
+hidden: true
+hideListLinks: true
+weight: 99
+---
+
+{{<warning>}}
+We do not recommend using RDS Proxy for RDI connections. The [Lambda function approach]({{< relref "/operate/rc/databases/rdi/setup#setup-lambda-function" >}}) provides better failover handling and is the recommended solution for production environments.
+
+Additionally, RDS Proxy does not work with RDS PostgreSQL and Aurora PostgreSQL because it does not support PostgreSQL logical replication.
+
+Only use RDS Proxy if you have specific requirements that necessitate it.
+{{</warning>}}
+
+## Overview
+
+RDS Proxy is a fully managed, highly available database proxy for Amazon RDS. While it can be used with RDI, we recommend the Lambda function approach instead for the following reasons:
+
+- **PostgreSQL incompatibility**: RDS Proxy does not support PostgreSQL logical replication, which is required for CDC (Change Data Capture).
+- **Added complexity**: RDS Proxy adds an additional layer between RDI and your database.
+- **Lambda provides better failover**: The Lambda function approach handles failover scenarios more efficiently.
+
+If you still need to use RDS Proxy, follow the instructions below.
+
+## Prerequisites
+
+Before setting up RDS Proxy, ensure you have:
+
+- An RDS or Aurora database (MySQL or SQL Server only)
+- AWS Secrets Manager secret containing your database credentials
+- AWS KMS encryption key for the secret
+- Appropriate IAM permissions
+
+## Create RDS Proxy
+
+Follow the AWS documentation to create an RDS Proxy:
+
+- [Creating an RDS Proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-setup.html) (AWS documentation)
+- [How RDS Proxy works](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.howitworks.html) (AWS documentation)
+- [RDS Proxy TLS/SSL](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy.howitworks.html#rds-proxy-security.tls) (AWS documentation)
+
+### IAM permissions
+
+The Proxy's IAM role must have the following permissions to access the database using the credentials secret and encryption key:
+
+- `secretsmanager:GetSecretValue`
+- `secretsmanager:DescribeSecret`
+- `kms:Decrypt`
+
+Example IAM policy:
+
+```json
+{
+    "Version": "2012-10-17",
+    "Statement": [
+        {
+            "Effect": "Allow",
+            "Action": [
+                "secretsmanager:GetSecretValue",
+                "secretsmanager:DescribeSecret"
+            ],
+            "Resource": "arn:aws:secretsmanager:region:account-id:secret:secret-name"
+        },
+        {
+            "Effect": "Allow",
+            "Action": "kms:Decrypt",
+            "Resource": "arn:aws:kms:region:account-id:key/key-id"
+        }
+    ]
+}
+```
+
+## Get the RDS Proxy IP address
+
+After creating the RDS Proxy, you need to get its static IP address to use when configuring the Network Load Balancer.
+
+To get the static IP address of your RDS Proxy, run the following command on an EC2 instance in the same VPC as the Proxy:
+
+```sh
+$ nslookup <proxy-endpoint>
+```
+
+Replace `<proxy-endpoint>` with the endpoint of your RDS Proxy. Save this IP address for use in the Network Load Balancer configuration.
+
+## Configure the Network Load Balancer
+
+When you [create the Network Load Balancer]({{< relref "/operate/rc/databases/rdi/setup#create-network-load-balancer-rds" >}}), use the RDS Proxy IP address instead of the database IP address:
+
+1. In **Register targets**, enter the static IP address of your RDS Proxy (obtained in the previous step).
+2. Enter the port number where your RDS Proxy is exposed.
+3. Select **Include as pending below**.
+4. Complete the remaining Network Load Balancer setup as described in the [main setup guide]({{< relref "/operate/rc/databases/rdi/setup#create-network-load-balancer-rds" >}}).
+
+## Next steps
+
+After setting up RDS Proxy and the Network Load Balancer:
+
+1. [Create an endpoint service]({{< relref "/operate/rc/databases/rdi/setup#create-endpoint-service-rds" >}}) through AWS PrivateLink.
+2. [Share your source database credentials]({{< relref "/operate/rc/databases/rdi/setup#share-source-database-credentials" >}}) with Redis Cloud.
+3. Continue with the [RDI pipeline configuration]({{< relref "/operate/rc/databases/rdi/define" >}}).
+
+{{<note>}}
+When using RDS Proxy, you do not need to set up the Lambda function for failover handling, as the proxy provides a static endpoint.
+{{</note>}}
+

content/operate/rc/databases/rdi/setup.md

@@ -130,33 +130,16 @@ For more details on AWS PrivateLink, see [Share your services through AWS Privat
 
 To set up PrivateLink for a database hosted on AWS RDS or AWS Aurora:
 
-{{<warning>}}
-The RDS Proxy does not work with RDS PostgreSQL and Aurora PostgreSQL because it does not support PostgreSQL logical replication.
+To connect to your RDS or Aurora database, we recommend using a Lambda function approach. This provides a reliable and secure connection method for all database types.
 
-For PostgreSQL databases, use one of the following alternatives instead:
-- **For test environments**: Connect the Network Load Balancer directly to the database IP address (skip the RDS Proxy step).
-- **For production environments**: Use the AWS Lambda approach described in [Access Amazon RDS across VPCs using AWS PrivateLink and Network Load Balancer](https://aws.amazon.com/blogs/database/access-amazon-rds-across-vpcs-using-aws-privatelink-and-network-load-balancer/).
-{{</warning>}}
-
-1. [Create an RDS Proxy](#create-rds-proxy) that will route requests to your database (MySQL and SQL Server only).
-1. [Create a network load balancer](#create-network-load-balancer-rds) that will route incoming requests to the RDS proxy (or directly to the database for PostgreSQL).
+1. [Create a network load balancer](#create-network-load-balancer-rds) that will route incoming requests to your database.
 1. [Create an endpoint service](#create-endpoint-service-rds) through AWS PrivateLink.
-
-### Create RDS proxy {#create-rds-proxy}
+1. [Set up Lambda function connectivity](#setup-lambda-function) to route requests to your database.
 
 {{<note>}}
-For RDS PostgreSQL and Aurora PostgreSQL, skip this step and proceed directly to [Create network load balancer](#create-network-load-balancer-rds) and configure it to connect directly to your database IP address.
+If you have specific requirements that necessitate using RDS Proxy instead of the recommended Lambda function approach, see the [RDS Proxy setup guide]({{< relref "/operate/rc/databases/rdi/rds-proxy" >}}). Note that RDS Proxy is not recommended and does not work with PostgreSQL.
 {{</note>}}
 
-In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Database** > **Aurora and RDS**. [Create an RDS proxy](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/rds-proxy-creating.html) that can access your database.
-
-The Proxy's IAM role must have the following permissions to access the database using the credentials secret and encryption key:
-- `secretsmanager:GetSecretValue`
-- `secretsmanager:DescribeSecret`
-- `kms:Decrypt`
-
-You can set the proxy's IAM role during creation in the **Authentication** section.
-
 ### Create network load balancer {#create-network-load-balancer-rds}
 
 In the [AWS Management Console](https://console.aws.amazon.com/), use the **Services** menu to locate and select **Compute** > **EC2**. [Create a network load balancer](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-network-load-balancer.html#configure-load-balancer) with the following settings:
@@ -166,31 +149,25 @@ In the [AWS Management Console](https://console.aws.amazon.com/), use the **Serv
     - **Load balancer IP address type**: Select **IPv4**.
 1. In **Network mapping**, select the VPC and availability zone associated with your source database.
 1. In **Security groups**, select the security group associated with your source database, or another security group that allows traffic from PrivateLink and allows traffic to the database.
-1. In **Listeners and routing**: 
+1. In **Listeners and routing**:
     1. Select **Create target group** to [create a target group](https://docs.aws.amazon.com/elasticloadbalancing/latest/application/create-target-group.html) with the following settings:
         1. In **Specify group details**:
             - **Target type**: Select **IP Addresses**.
             - **Protocol : Port**: Select **TCP**, and then enter the port number where your database is exposed.
             - The **IP address type** and **VPC** should be selected already and match the VPC you selected earlier.
-        1. In **Register targets**, enter the static IP address of your RDS proxy (for MySQL and SQL Server) or your database (for PostgreSQL), enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return to **Listeners and routing** in the Network Load Balancer setup.
+        1. In **Register targets**, enter the static IP address of your database, enter the port, and select **Include as pending below**. Then, select **Create target group** to create your target group. Return to **Listeners and routing** in the Network Load Balancer setup.
 
-            **For MySQL and SQL Server**: To get the static IP address of your RDS Proxy, run the following command on an EC2 instance in the same VPC as the Proxy:
-            ```sh
-            $ nslookup <proxy-endpoint>
-            ```
-            Replace `<proxy-endpoint>` with the endpoint of your RDS proxy.
-
-            **For PostgreSQL**: To get the static IP address of your database, run the following command on an EC2 instance in the same VPC as the database:
+            To get the static IP address of your database, run the following command on an EC2 instance in the same VPC as the database:
             ```sh
             $ nslookup <database-endpoint>
             ```
-            Replace `<database-endpoint>` with the endpoint of your RDS or Aurora PostgreSQL database.
+            Replace `<database-endpoint>` with the endpoint of your RDS or Aurora database.
     1. Set the following **Listener** properties:
         - **Protocol**: Select **TCP**.
         - **Port**: Enter your source database's port.
         - **Default action**: Select the target group you created in the previous step.
 1. Review the network load balancer settings, and then select **Create load balancer** to continue.
-1. After the network load balancer is active, select **Security**. 
+1. After the network load balancer is active, select **Security**.
 
     If you selected the same security group as your source database, you must not enforce security group rules on PrivateLink traffic. Select **Edit** and then deselect **Enforce inbound rules on PrivateLink traffic**, and then select **Save changes**.
 
@@ -224,6 +201,102 @@ After you create the endpoint service, you need to add Redis Cloud as an Allowed
 
 For more details on AWS PrivateLink, see [Share your services through AWS PrivateLink](https://docs.aws.amazon.com/vpc/latest/privatelink/privatelink-share-your-services.html).
 
+### Set up Lambda function connectivity {#setup-lambda-function}
+
+{{<note>}}
+Setting up the Lambda function is optional but recommended for production environments. The Lambda function provides automatic failover handling and a more robust connection to your RDS or Aurora database.
+{{</note>}}
+
+The Lambda function monitors RDS failover events and automatically updates the NLB Target Group to point to the new primary instance's IP address. This ensures RDI reconnects automatically after a failover.
+
+#### Option 1: Use the Redis Terraform module
+
+Redis provides a ready-to-use Terraform module that automates the Lambda function deployment. This is the recommended approach.
+
+##### Prerequisites
+
+- [Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) >= 1.5.7
+- [AWS CLI](https://aws.amazon.com/cli/) configured with credentials
+- The ARNs from the Network Load Balancer and Endpoint Service you created in the previous steps
+
+##### Required variables
+
+Before deploying the Lambda module, gather the following information:
+
+| Variable | Description | Where to find it |
+|----------|-------------|------------------|
+| `identifier` | A unique name for the Lambda resources | Choose a descriptive name (e.g., `rdi-failover-handler`) |
+| `db_endpoint` | Your RDS cluster or instance endpoint | AWS Console → RDS → Your database → Connectivity |
+| `db_port` | Your database port | AWS Console → RDS → Your database → Connectivity (default: `5432` for PostgreSQL, `3306` for MySQL, `1433` for SQL Server) |
+| `elb_tg_arn` | The NLB Target Group ARN | AWS Console → EC2 → Target Groups → Your target group |
+| `rds_arn` | The RDS cluster or instance ARN | AWS Console → RDS → Your database → Configuration |
+| `rds_cluster_identifier` | The RDS cluster identifier | AWS Console → RDS → Your cluster name |
+
+##### Deploy the Lambda module
+
+1. Clone the Redis cloud automation repository:
+
+    ```bash
+    git clone https://github.com/redis/rdi-cloud-automation.git
+    cd rdi-cloud-automation/modules/aws-rds-lambda
+    ```
+
+1. Create a `terraform.tfvars` file with your configuration:
+
+    ```hcl
+    identifier             = "rdi-failover-handler"
+    db_endpoint            = "your-cluster.cluster-xxxxxxxxx.us-east-1.rds.amazonaws.com"
+    db_port                = 5432
+    elb_tg_arn             = "arn:aws:elasticloadbalancing:us-east-1:123456789012:targetgroup/your-tg/xxxxxxxxx"
+    rds_arn                = "arn:aws:rds:us-east-1:123456789012:cluster:your-cluster"
+    rds_cluster_identifier = "your-cluster"
+    ```
+
+1. Initialize and apply Terraform:
+
+    ```bash
+    terraform init
+    terraform apply
+    ```
+
+##### How the Lambda function works
+
+The deployed Lambda function:
+
+1. **Monitors RDS events**: Subscribes to RDS failover events via SNS
+1. **Detects failover**: When a failover occurs, RDS triggers an SNS notification
+1. **Resolves new IP**: The Lambda function queries DNS to get the new primary's IP address
+1. **Updates NLB target**: Automatically updates the NLB Target Group with the new IP
+
+This process typically completes within 30-60 seconds for Aurora, or 60-120 seconds for standard RDS.
+
+##### Verify the deployment
+
+After deployment, verify the Lambda function is configured correctly:
+
+1. Check the Lambda function in AWS Console → Lambda → Functions
+1. Verify the environment variables are set correctly:
+    - `Cluster_EndPoint`: Your RDS endpoint
+    - `RDS_Port`: Your database port
+    - `NLB_TG_ARN`: Your NLB Target Group ARN
+1. Check the SNS subscription in AWS Console → SNS → Subscriptions
+
+#### Option 2: Full infrastructure deployment
+
+For new deployments, Redis provides a complete Terraform example that deploys the entire infrastructure including the RDS database, NLB, PrivateLink, and Lambda function.
+
+See the [AWS RDS PrivateLink Failover Example](https://github.com/redis/rdi-cloud-automation/tree/main/examples/aws-rds-privatelink-failover) for:
+
+- Multi-engine support (PostgreSQL, MySQL, SQL Server)
+- Automatic CDC user creation
+- Complete VPC and networking setup
+- Lambda-based failover handling
+
+#### Option 3: Manual Lambda setup
+
+For custom implementations, refer to the AWS documentation:
+[Access Amazon RDS across VPCs using AWS PrivateLink and Network Load Balancer](https://aws.amazon.com/blogs/database/access-amazon-rds-across-vpcs-using-aws-privatelink-and-network-load-balancer/)
+
 {{< /multitabs >}}
 
 ## Share source database credentials