
subaud is authored by Court Schuett (he/him). While Court is an AWS employee, opinions and statements on this blog are his own.


Court Schuett

# Building a Private Fargate Task and Lambda

In order to build an AWS Fargate task in a [private subnet](https://docs.aws.amazon.com/vpc/latest/userguide/configure-subnets.html) that can still be invoked by a Lambda function, the Lambda function must also be built in the same VPC with access to the private subnet.

![Overview](/images/blog/PrivateLambdaFargate.png)

## GitHub Repo

Full code is available here: https://github.com/schuettc/cdk-private-fargate-with-lambda

## Building the VPC and ALB

```typescript
this.vpc = new Vpc(this, 'VPC', {
  subnetConfiguration: [
    {
      cidrMask: 24,
      name: 'PrivateSubnet',
      subnetType: SubnetType.PRIVATE_WITH_EGRESS,
    },
    {
      cidrMask: 24,
      name: 'PublicSubnet',
      subnetType: SubnetType.PUBLIC,
    },
  ],
  maxAzs: 2,
});
```

Here we are creating a VPC with two types of Subnets - `PRIVATE_WITH_EGRESS` and `PUBLIC`.

```typescript
this.applicationLoadBalancer = new ApplicationLoadBalancer(this, 'alb', {
  vpc: this.vpc,
  vpcSubnets: { subnetType: SubnetType.PRIVATE_WITH_EGRESS },
  internetFacing: false,
  securityGroup: this.fargateSecurityGroup,
});
```

Next we will create an Application Load Balancer in the `PRIVATE_WITH_EGRESS` subnet.

## Fargate

```typescript
this.fargateService = new ApplicationLoadBalancedFargateService(
  this,
  'fargateService',
  {
    taskImageOptions: {
      image: ContainerImage.fromRegistry('amazon/amazon-ecs-sample'),
      taskRole: fargateTaskRole,
    },
    publicLoadBalancer: false,
    vpc: props.vpc,
    assignPublicIp: false,
    openListener: false,
    loadBalancer: props.applicationLoadBalancer,
    listenerPort: 80,
    taskSubnets: {
      subnetType: SubnetType.PRIVATE_WITH_EGRESS,
    },
    securityGroups: [props.fargateSecurityGroup],
    runtimePlatform: {
      operatingSystemFamily: OperatingSystemFamily.LINUX,
      cpuArchitecture: CpuArchitecture.X86_64,
    },
  },
);
```

Next we will create the Fargate Service and Task, once again assigning it to the `PRIVATE_WITH_EGRESS` subnet using the previously created Application Load Balancer.

## Lambda

```typescript
const fargateLambdaInPrivateVPCwithSG = new NodejsFunction(
  this,
  'fargateLambdaInPrivateVPCwithSG',
  {
    entry: 'src/resources/fargateLambda/index.ts',
    runtime: Runtime.NODEJS_LATEST,
    architecture: Architecture.ARM_64,
    role: lambdaRole,
    timeout: Duration.seconds(60),
    vpc: props.vpc,
    vpcSubnets: { subnetType: SubnetType.PRIVATE_WITH_EGRESS },
    securityGroups: [lambdaSecurityGroup],
    environment: {
      FARGATE_ALB_URL: props.fargateService.loadBalancer.loadBalancerDnsName,
    },
  },
);
```

Finally, we will create the Lambda function and assign to the `PRIVATE_WITH_EGRESS` subnet.

## Security Groups

In addition to containing the Lambda, Application Load Balancer, and Fargate within the same VPC and private subnet, Security Groups can also be used to restrict access.

```typescript
this.fargateSecurityGroup = new SecurityGroup(this, 'fargateSecurityGroup', {
  vpc: this.vpc,
  description: 'Security Group for Fargate ALB',
});
```

First we will create a Security Group for the Application Load Balancer.

```typescript
const lambdaSecurityGroup = new SecurityGroup(this, 'LambdaSecurityGroup', {
  vpc: props.vpc,
});
```

Next we will create a Security Group for the Lambda function.

```typescript
props.fargateSecurityGroup.connections.allowFrom(
  new Connections({
    securityGroups: [lambdaSecurityGroup],
  }),
  Port.tcp(80),
  'allow traffic on port 80 from the Lambda security group',
);
```

Finally, we will allow a connection from the Lambda Security Group to the Application Load Balancer Security Group. This will allow the Lambda function to make requests to the Fargate Task through the Application Load Balancer. The other two Lambda functions show how this is required as they do not work. The first does not include the correct Security Group and the second does not include the correct VPC subnet.

## Result

Once deployed, the Lambda function in the correct VPC with the correct Security Group will be able to make a request to the Fargate task and get a response:

```html
<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
    <title>Simple PHP App</title>
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link href="assets/css/bootstrap.min.css" rel="stylesheet" />
    <style>
      body {
        margin-top: 40px;
        background-color: #333;
      }
    </style>
    <link href="assets/css/bootstrap-responsive.min.css" rel="stylesheet" />
    
  </head>

  <body>
    <div class="container">
      <div class="hero-unit">
        <h1>Simple PHP App</h1>
        <h2>Congratulations</h2>
        <p>Your PHP application is now running on a container in Amazon ECS.</p>
        <p>The container is running PHP version 5.4.16.</p>
      </div>
    </div>
  </body>
</html>
```

The other Lambda functions will timeout and fail

```text
Task timed out after 60.07 seconds
```

Additionally, the Application Load Balancer URL will only return an RFC 1918 private address:

```bash
$ nslookup internal-Privat-vpcRe-8uesdCpJs6j6-1716194524.us-east-1.elb.amazonaws.com

Non-authoritative answer:
Name:	internal-privat-vpcre-8uesdcpjs6j6-1716194524.us-east-1.elb.amazonaws.com
Address: 10.0.1.253
Name:	internal-privat-vpcre-8uesdcpjs6j6-1716194524.us-east-1.elb.amazonaws.com
Address: 10.0.0.230
```

How to build a private Fargate task and connect to it with an AWS Lambda function using AWS CDK.  

Using CDK to build a Fargate task with AWS Lambda integration


In this blog, we will walk through building an [Amazon RDS](https://aws.amazon.com/rds/) database using CDK. This database will be built in an isolated subnet. An AWS Lambda function will be built to initialize this database during the deployment of the CDK. Another AWS Lambda function will be built to write data to this database once a day. An EC2 instance will be built to connect to this database.

## GitHub Repo

Full code is available here: https://github.com/schuettc/cdk-private-rds-with-lambda

## VPC - Virtual Private Cloud

The first Construct built in this CDK is the Amazon Virtual Private Cloud (VPC). This will lay the foundation for future networking and placement of infrastructure.

```typescript
this.vpc = new Vpc(this, 'VPC', {
  natGateways: 1,
  subnetConfiguration: [
    {
      cidrMask: 24,
      name: 'Private',
      subnetType: SubnetType.PRIVATE_ISOLATED,
    },
    {
      cidrMask: 24,
      name: 'PrivateWithEgress',
      subnetType: SubnetType.PRIVATE_WITH_EGRESS,
    },
    {
      cidrMask: 24,
      name: 'Public',
      subnetType: SubnetType.PUBLIC,
    },
  ],
});
```

We will be building a VPC with three SubnetTypes. This Construct will build a complete VPC infrastructure, including:

- Multi-AZ Subnets
- Route Tables
- Security Groups
- Internet Gateway
- NAT Gateway

We will be using this infrastructure for our EC2 instance, RDS database, and Lambda functions.

## PostgreSQL RDS Database

Once the VPC is created, we will be creating an RDS Database within it. In this example, we will be creating a PostgreSQL database.

```typescript
this.database = new DatabaseInstance(this, 'database', {
  engine: DatabaseInstanceEngine.POSTGRES,
  vpc: props.vpc,
  vpcSubnets: { subnetType: SubnetType.PRIVATE_ISOLATED },
  instanceType: InstanceType.of(
    InstanceClass.BURSTABLE4_GRAVITON,
    InstanceSize.LARGE,
  ),
  multiAz: false,
  allowMajorVersionUpgrade: true,
  autoMinorVersionUpgrade: true,
  backupRetention: Duration.days(21),
  securityGroups: [props.securityGroup],
});
```

Because we use `vpcSubnets: { subnetType: SubnetType.PRIVATE_ISOLATED }` this RDS instance will be created in a [subnet](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.SubnetType.html#private_isolated) that does not route traffic to the Internet. To connect to this RDS database, we will have to create resources within the VPC that can connect to it.

## Database Initializer Custom Resource

Once the PostgreSQL database has been created, we need to initialize it. This will be done using a Custom Resource during the deployment of the CDK. This Custom Resource will be an AWS Lambda function that creates a table. This AWS Lambda Function must be created in the same VPC as the RDS database.

```typescript
const initializeLambda = new Function(this, 'InitializeTableLambda', {
  code: Code.fromAsset(path.join(__dirname, 'resources/initialize_lambda'), {
    bundling: {
      image: Runtime.PYTHON_3_9.bundlingImage,
      command: [
        'bash',
        '-c',
        'pip install -r requirements.txt -t /asset-output && cp -au . /asset-output',
      ],
    },
  }),
  runtime: Runtime.PYTHON_3_9,
  vpc: props.vpc,
  vpcSubnets: { subnetType: SubnetType.PRIVATE_WITH_EGRESS },
  architecture: Architecture.ARM_64,
  layers: [props.powerToolsLayer],
  role: props.role,
  handler: 'index.handler',
  timeout: Duration.minutes(5),
  environment: {
    RDS_SECRET_NAME: props.dataBase.secret?.secretName!,
  },
});
```

This AWS Lambda Function will be used by the Custom Resource to initialize the PostgreSQL database during the creation of the CDK.

### Database Initializer Function

```python
def create_table():
    create_table = """CREATE TABLE queries(
            id SERIAL PRIMARY KEY,
            query_date DATE
            )
            """
    try:
        cursor.execute(create_table)
        cursor.close()
        connection.commit()
    except (Exception, psycopg2.DatabaseError) as error:
        print(error)
    finally:
        if connection is not None:
            connection.close()
```

When this AWS Lambda Function is called during the deployment of the CDK, we will create a table with two columns: `id` and `query_date`. These columns will be used by the recurring Query function to populate the RDS database.

## EC2 Instance

In order to interact with the PostgreSQL database, we will be creating an EC2 instance that is able to connect to the database.

```typescript
const ec2Instance = new Instance(this, 'Instance', {
  vpc: props.vpc,
  vpcSubnets: { subnetType: SubnetType.PRIVATE_WITH_EGRESS },
  instanceType: InstanceType.of(InstanceClass.C6G, InstanceSize.MEDIUM),
  machineImage: ami,
  init: CloudFormationInit.fromConfigSets({
    configSets: {
      default: ['install'],
    },
    configs: {
      install: new InitConfig([
        InitCommand.shellCommand('yum update -y'),
        InitCommand.shellCommand('yum upgrade -y'),
        InitCommand.shellCommand('amazon-linux-extras install postgresql15'),
      ]),
    },
  }),
  initOptions: {
    timeout: Duration.minutes(20),
    includeUrl: true,
    includeRole: true,
    printLog: true,
  },
  role: ec2Role,
});
```

This instance will be created in the same `PRIVATE_WITH_EGRESS` SubnetType that the AWS Lambda Functions are created in. This will allow us to install the [`psql` client](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToPostgreSQLInstance.html#USER_ConnectToPostgreSQLInstance.psql) and connect to the database. This instance can be used only as needed and stopped when not needed to reduce EC2 costs.

From the EC2 Console, you can Connect to the instance.

![EC2Instance](/images/blog/RDS-EC2Instance.png)

### Secret Manager

In order to connect to the PostgreSQL database, you will need the `hostname`, `username`, and `password`. These have been automatically generated for you and stored in [AWS Secrets Manager](https://aws.amazon.com/secrets-manager/).

![SecretManger](/images/blog/RDS-SecretManager.png)

Once connected to the the EC2 Instance, using [AWS Session Manger](https://docs.aws.amazon.com/systems-manager/latest/userguide/session-manager.html) you can connect to the RDS database using `psql`.

```bash
psql -h host -p 5432 -U postgres postgres
```

In this example, replace `host` with the value from the Secrets.

![Secrets](/images/blog/RDS-Secrets.png)

At the prompt, enter the `password` from the Secrets.

### PostgreSQL Verification

From the prompt, verify that the table has been created:

```sql
\dt+
```

Output:

```
                      List of relations
 Schema |  Name   | Type  |  Owner   |  Size   | Description
--------+---------+-------+----------+---------+-------------
 public | queries | table | postgres | 0 bytes |
```

However, nothing has been created in the table yet:

```sql
SELECT * FROM queries;
```

Output:

```
 id | query_date
----+------------
(0 rows)
```

### Writing Data

To write data to the PostgreSQL database, you can wait for the EventBridge scheduled rule to trigger the AWS Lambda Function, or you can test the function manually. Once run, the AWS Lambda function should write a single record to the file that can be seen by using:

```SQL
SELECT * FROM queries;
```

With expected output similar to:

```
 id | query_date
----+------------
  1 | 2022-11-07
(1 row)
```

## Conclusion

Instructions for deploying this demo are available at the GitHub repo. Once deployed, you should now have an Amazon RDS database deployed to an isolated subnet that is able to be used by both an EC2 Instance and AWS Lambda function. This will allow you to update and interact with the PostgreSQL database while ensuring that it is in an isolated subnet. In future posts, we will explore how this database can be used with Amazon Quicksight to create customized reports and dashboards.


How to build a private RDS database and connect to it with an AWS Lambda function using AWS CDK.  

Vpc

Using CDK to build a Fargate task with AWS Lambda integration

Using CDK to build an RDS database with AWS Lambda integration