Skip to content


This is a reference design and implementation of an environment template to provision a self-service environment with a Generative AI application running on Amazon ECS with a LLM from Amazon Bedrock. This reference design is based on Rafay's Environment Manager.

The ECS cluster is configured to use the "Fargate Launch Type Model" which makes the environment extremely simple and cost effective. AWS Fargate allows Amazon ECS to run containers without having to manage servers or clusters of Amazon EC2 instances. With Fargate, you no longer have to provision, configure, or scale clusters of virtual machines to run containers. This removes the need to choose server types, decide when to scale your clusters, or optimize cluster packing.

Organizations also want developer environments to be extremely cost effective. With Fargate and ECS,

  • You only pay for the amount of vCPU and memory resources that your containerized application requests.
  • vCPU and memory resources are calculated from the time your container images are pulled until the Amazon ECS Task terminates, rounded up to the nearest second.


Source Code for Template

The Terraform source code and related assets are available in a public Git repo as shown below

Git Repo

Initial Setup

The platform team is expected to perform an initial configuration and setup of the Amazon ECS based Environment Template. The sequence diagram describes the high level steps. In this step, the platform team will create the Environment Template in a root level, central project they control and will share the template downstream with every developer.

    participant plat as Platform Team
    participant rafay as Rafay <br> Environment Manager
    participant csp as AWS 

    plat->>rafay: Create Central "genai" Project 
    plat->>rafay: Import Gen AI <br>Environment Template for ECS 
    plat->>csp: Enable Models in Amazon Bedrock 
    plat-->>rafay: Validate Template by <br>creating sample environment (optional)

Request for Environment

When a developer requests for a Gen AI environment (based on ECS), a series of automated workflows will need to be performed to fulfil this request. The sequence diagram below describes the high level steps.


Although the recommended workflow assumes and recommends using an Integration with an Identity Provider (IdP) to provide a Single Sign On (SSO) experience, organizations can also use locally managed users.

    participant dev as Developer
    participant plat as Platform Team 
    participant rafay as Rafay <br> Environment Manager
    participant idp as Identity Provider <br> (IdP)

    dev->>plat: Request Gen AI Environment <br>based on ECS 
    plat-->>rafay: Create New Project in Org <br> for Developer
    rafay->>idp: Create Group for New Project  
    rafay->>idp: Add developer to Group 
    rafay-->>rafay: Create Group in Rafay <br>with Env Template User Role 
    rafay->>rafay: Share "genai" Env Template <br>with New Project 
    rafay-->>dev: Environment Ready for Use 

Developer Creates Environment

The developer will have the ability to deploy and deprovision environments based on the shared environment template. Note that the developer

  • Does not need to have any knowledge of Terraform
  • Does not need access to privileged credentials for AWS
  • Does not any help from the Platform team to deploy their environment
    participant dev as Developer
    participant rafay as Rafay <br> Environment Manager
    participant csp as ECS Cluster
    participant idp as Identity Provider 

    dev->>idp: Access Environment 
    idp-->>dev: Redirect to Rafay 
    dev-->>rafay: SSO to Rafay with <br> RBAC (Env Template User)

    dev->>rafay: Create Environment <br>based on Env Template 
    rafay->>csp: Provision new ECS Cluster w/VPC, subnets and Gen AI App  
    rafay-->>dev: Environment Ready 
    dev->>csp: Uses GenAI Environment 


  • You have access to a AWS Account
  • You have access to a Mac or Linux machine
  • You have a Git client on your machine that is setup for push/pull
  • You have Docker installed on your machine