(rds): support for Aurora Serverless V2

[rpbarnes]

Describe the feature

Please like the original post instead of leaving a +1 comment.

Add CDK support for aurora serverless v2 ideally via the ServerlessCluster construct.

edit there's a few solutions using base cloud formation constructs in the comments. Please see those for a work around.

Currently deploying a Serverless V2 instance via cdk doesn't seem possible.

I've got this far

        enum ServerlessInstanceType {
            SERVERLESS = 'serverless',
        }
        type CustomInstanceType = ServerlessInstanceType | ec2.InstanceType;
        const CustomInstanceType = {
            ...ServerlessInstanceType,
            ...ec2.InstanceType,
        };

        this.serverlessCluster = new DatabaseCluster(
            this,
            'ServerlessClusterV2',
            {
                engine: DatabaseClusterEngine.auroraMysql({
                    version: AuroraMysqlEngineVersion.of(
                        '8.0.mysql_aurora.3.02.0'
                    ), // The new minor version of Database Engine.
                }),
                storageEncrypted: true,
                iamAuthentication: true,
                parameterGroup: ParameterGroup.fromParameterGroupName(
                    this,
                    'rdsClusterPrameterGroup',
                    'default.aurora-mysql8.0'
                ),
                storageEncryptionKey: new Key(this, 'dbEncryptionKey'),
                instanceProps: {
                    instanceType:
                        CustomInstanceType.SERVERLESS as unknown as InstanceType,
                    vpc: props.vpc,
                    vpcSubnets: {
                        subnetType: SubnetType.PRIVATE_ISOLATED,
                    },
                },
            }
        );

but I keep running into this error

Set the Serverless v2 scaling configuration on the parent DB cluster before creating a Serverless v2 DB instance.

Which I can't figure out a way around.

Use Case

Need to create a DB via CDK. I want to use Serverless V2 because of the support for mysql 8.0.

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.15.0

Environment details (OS name and version, etc.)

mac OS X

[jvlch]

I found a workaround using a custom resource, uses addDependency on instances to a custom resource that sets the min/max capacity settings

enum ServerlessInstanceType {
  SERVERLESS = 'serverless',
}

type CustomInstanceType = ServerlessInstanceType | ec2.InstanceType;

const CustomInstanceType = { ...ServerlessInstanceType, ...ec2.InstanceType };

const dbClusterInstanceCount: number = 1

const dbCluster = new rds.DatabaseCluster(this, 'DbCluster', {
  engine: rds.DatabaseClusterEngine.auroraPostgres({
    version: rds.AuroraPostgresEngineVersion.VER_13_6,
  }),
  instances: dbClusterInstanceCount,
  instanceProps: {
    vpc: vpc,
    instanceType: CustomInstanceType.SERVERLESS as unknown as ec2.InstanceType,
    autoMinorVersionUpgrade: false,
    publiclyAccessible: false,
    securityGroups: [
      dbSecurityGroup,
    ],
    vpcSubnets: {
      subnets: [vpc.isolatedSubnets[0], vpc.isolatedSubnets[1], vpc.isolatedSubnets[2]]
    },
  },
  credentials: rds.Credentials.fromSecret(dbAdminSecret),
  backup: {
    retention: cdk.Duration.days(7),
    preferredWindow: '08:00-09:00'
  },
  port: 5432,
  cloudwatchLogsExports: ["postgresql"],
  cloudwatchLogsRetention: logs.RetentionDays.SIX_MONTHS,
  subnetGroup: dbSubnetGroup,
  storageEncrypted: true,
  storageEncryptionKey: dbKey,
})


const serverlessV2ScalingConfiguration = {
  MinCapacity: 0.5,
  MaxCapacity: 16,
}

const dbScalingConfigure = new cr.AwsCustomResource(this, 'DbScalingConfigure', {
  onCreate: {
    service: "RDS",
    action: "modifyDBCluster",
    parameters: {
      DBClusterIdentifier: dbCluster.clusterIdentifier,
      ServerlessV2ScalingConfiguration: serverlessV2ScalingConfiguration,
    },
    physicalResourceId: cr.PhysicalResourceId.of(dbCluster.clusterIdentifier)
  },
  onUpdate: {
    service: "RDS",
    action: "modifyDBCluster",
    parameters: {
      DBClusterIdentifier: dbCluster.clusterIdentifier,
      ServerlessV2ScalingConfiguration: serverlessV2ScalingConfiguration,
    },
    physicalResourceId: cr.PhysicalResourceId.of(dbCluster.clusterIdentifier)
  },
  policy: cr.AwsCustomResourcePolicy.fromSdkCalls({
    resources: cr.AwsCustomResourcePolicy.ANY_RESOURCE,
  }),
})

const cfnDbCluster = dbCluster.node.defaultChild as rds.CfnDBCluster
const dbScalingConfigureTarget = dbScalingConfigure.node.findChild("Resource").node.defaultChild as cdk.CfnResource

cfnDbCluster.addPropertyOverride("EngineMode", "provisioned")
dbScalingConfigure.node.addDependency(cfnDbCluster)


for (let i = 1 ; i <= dbClusterInstanceCount ; i++) { 
  (dbCluster.node.findChild(`Instance${i}`) as rds.CfnDBInstance).addDependsOn(dbScalingConfigureTarget)
}

[rpbarnes]

Wo that's cool.

I'll give that a try. Just curious though... Is AWS planning on adding Aurora Serverless V2 setup as proper cdk constructs? I'm happy to wait a bit if I can get the proper thing instead of moving forward with the workaround.

[skinny85]

Yes, this is definitely on our roadmap. Not sure when we're going to get to it though - leaving +1s on the issue will definitely help us prioritize.

We also encourage community contributions. Our "Contributing" guide: https://github.com/aws/aws-cdk/blob/master/CONTRIBUTING.md.

Thanks,
Adam

[fourgates]

+1

[rpbarnes]

+1

…

Sent from my iPhone

On May 4, 2022, at 7:47 PM, Phillip Ninan ***@***.***> wrote:  +1 — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[carlsonorozco]

+1

[owenmorgan]

+1

[valentin-nasta]

+1

[jvlch]

+1

[alexandrustf]

+1

[Anonyfox]

+1

[code11octo]

+1

[therockstorm]

I'm happy to see this is such a popular request as I want it too! Could we please 👍 the original description instead of adding +1 comments? I imagine there are many people subscribed to this issue and each of these comments notify all of them. Additionally, adding a 👍 allows the CDK team to sort issues by that reaction to inform priority.

[RubysDad]

+1

[Harper04]

+1

[mrgum]

+1

[JeromeDuboisVizzia]

+1

[guywilsonjr]

I'm not sure why no one has mentioned this yet, but you could always use something like:

self.creds = ...
self.cluster_engine = ...
self.vpc = ...

self.cluster = rds.DatabaseCluster(
            self,
            "Cluster",
            instances=1,
            credentials=self.creds,
            engine=self.cluster_engine,
            instance_props=rds.InstanceProps(
                instance_type=InstanceType('serverless'),
                vpc=self.vpc,
                vpc_subnets=ec2.SubnetSelection(
                    subnet_type=ec2.SubnetType.PUBLIC
                ),
                delete_automated_backups=True))
        cluster_node: rds.CfnDBCluster = self.cluster.node.default_child
        cluster_node.add_property_override('ServerlessV2ScalingConfiguration', {"MinCapacity": 1, "MaxCapacity": 16})

[DEADSEC-SECURITY]

That works very well @guywilsonjr but of course a proper L1 or L2 needs to be developed by the devs.

[corymhall]

Just to give an update to everyone I am actively working on this and I will provide updates as I make progress.

[kylerjensen]

I was able to patch this functionality into the existing L2 construct using patch-package. This solution appears to be working well for me. @corymhall you may want to take a look at this. Here was the patch file I added to my patches directory:

aws-cdk-lib+2.77.0.patch

Edit: This no longer works as of [email protected]

[mccauleyp]

+1

[ShaneHudson]

@corymhall I'm curious to how you intend to solve the potential "chicken and egg" problem with Aurora v2 not having the Data API? We are using this to run database migrations after the DB has been successfully deployed. I feel like this is a problem most (if not all) people will encounter when running v2 in CDK? The CDK will synth the cloud formation files and deploy them, but will need to after successful build run the migrations.

Is this just an issue due to my team's antiquated pipelines or is it something you think many people will have trouble with?

[Lewenhaupt]

@corymhall I'm curious to how you intend to solve the potential "chicken and egg" problem with Aurora v2 not having the Data API? We are using this to run database migrations after the DB has been successfully deployed. I feel like this is a problem most (if not all) people will encounter when running v2 in CDK? The CDK will synth the cloud formation files and deploy them, but will need to after successful build run the migrations.

Is this just an issue due to my team's antiquated pipelines or is it something you think many people will have trouble with?

@ShaneHudson We use custom resources for this. They share the same function but each run one migration file each. The custom resources depend on one another which ensures they are ran in the correct order, and the first depends on the Aurora v2 instance. Works pretty well.

[moltar]

@corymhall I'm curious to how you intend to solve the potential "chicken and egg" problem with Aurora v2 not having the Data API? We are using this to run database migrations after the DB has been successfully deployed. I feel like this is a problem most (if not all) people will encounter when running v2 in CDK? The CDK will synth the cloud formation files and deploy them, but will need to after successful build run the migrations.

Is this just an issue due to my team's antiquated pipelines or is it something you think many people will have trouble with?

I think arguably migrations cross the line of "infra" into the application domain.

So they need to be treated separately.

But we also mix the concerns a bit here. The way we run migrations is by triggering a CodeBuild project, which runs the migrations at the deployment time. Migration files are packaged up using Bucket Deployment, and then CodeBuild uses S3 as source.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.