This is a reference design and implementation of an environment template to provision a self-service environment with a Generative AI application running on Amazon ECS with a LLM from Amazon Bedrock. This reference design is based on Rafay's Environment Manager.
The ECS cluster is configured to use the "Fargate Launch Type Model" which makes the environment extremely simple and cost effective. AWS Fargate allows Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances. With Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers. This removes the need to choose server types, decide when to scale your clusters, or optimize cluster packing.
Organizations also want developer environments to be extremely cost effective. With Fargate and ECS,
- You only pay for the amount of vCPU and memory resources that your containerized application requests.
- vCPU and memory resources are calculated from the time your container images are pulled until the Amazon ECS Task terminates, rounded up to the nearest second.
Source Code for Template¶
The Terraform source code and related assets are available in a public Git repo as shown below
The platform team is expected to perform an initial configuration and setup of the Amazon ECS based Environment Template. The sequence diagram describes the high level steps. In this step, the platform team will create the Environment Template in a root level, central project they control and will share the template downstream with every developer.
sequenceDiagram participant plat as Platform Team participant rafay as Rafay <br> Environment Manager participant csp as AWS plat->>rafay: Create Central "genai" Project plat->>rafay: Import Gen AI <br>Environment Template for ECS plat->>csp: Enable Models in Amazon Bedrock plat-->>rafay: Validate Template by <br>creating sample environment (optional)
Request for Environment¶
When a developer requests for a Gen AI environment (based on ECS), a series of automated workflows will need to be performed to fulfil this request. The sequence diagram below describes the high level steps.
Although the recommended workflow assumes and recommends using an Integration with an Identity Provider (IdP) to provide a Single Sign On (SSO) experience, organizations can also use locally managed users.
sequenceDiagram participant dev as Developer participant plat as Platform Team participant rafay as Rafay <br> Environment Manager participant idp as Identity Provider <br> (IdP) dev->>plat: Request Gen AI Environment <br>based on ECS plat-->>rafay: Create New Project in Org <br> for Developer rafay->>idp: Create Group for New Project rafay->>idp: Add developer to Group rafay-->>rafay: Create Group in Rafay <br>with Env Template User Role rafay->>rafay: Share "genai" Env Template <br>with New Project rafay-->>dev: Environment Ready for Use
Developer Creates Environment¶
The developer will have the ability to deploy and deprovision environments based on the shared environment template. Note that the developer
- Does not need to have any knowledge of Terraform
- Does not need access to privileged credentials for AWS
- Does not any help from the Platform team to deploy their environment
sequenceDiagram participant dev as Developer participant rafay as Rafay <br> Environment Manager participant csp as ECS Cluster participant idp as Identity Provider dev->>idp: Access Environment idp-->>dev: Redirect to Rafay dev-->>rafay: SSO to Rafay with <br> RBAC (Env Template User) dev->>rafay: Create Environment <br>based on Env Template rafay->>csp: Provision new ECS Cluster w/VPC, subnets and Gen AI App rafay-->>dev: Environment Ready dev->>csp: Uses GenAI Environment
- You have access to a AWS Account
- You have access to a Mac or Linux machine
- You have a Git client on your machine that is setup for push/pull
- You have Docker installed on your machine