White Listing-GPU
This section is relevant for customers that have been allocated a whitelabeled environment for Rafay's GPU Platform using Rafay's Hosted Control Plane for GPU PaaS use cases.
This environment is generally provided to customers evaluating the Rafay GPU Platform's capabilities and do not wish to invest the resources (i.e. infrastructure and time) to install and operate a self hosted version of the Rafay control plane.
Customers can use the information in this page to whitelist inbound (emails) and outbound (control channel destinations) in their datacenter firewalls and proxies.
Note that none of this applies when customers deploy and operate a self hosted Rafay control plane in their datacenter.
Info
If you are enterprise that has been provisioned with an Org in https://console.rafay.dev, this page does not apply to you. Please navigate to this page instead.
Emails¶
Rafay uses a 3rd party email deliver service to send emails to users. Emails are sent to users of the platform for first time user activation, password resets, notifications etc. To guarantee delivery of emails, we strongly recommend that customers "whitelist" the IP address used for sending emails in their inbound email security systems.
The dedicated IP address currently used for sending emails is 149.72.39.92
Network Firewall¶
Rafay's hosted control plane for GPUs has been designed to manage and orchestrate infrastrastructure in the customer's datacenter operating behind a firewall. Customers will deploy a Rafay orchestration agent in each datacenter (behind their firewall). The agent will establish and maintain a long-running outbound TLS based control channel.
Info
No inbound ports need to be opened on the customer's firewall.
Outbound Ports¶
The Rafay orchestration agent deployed in the customer's datacenter requires "TCP Port 443, Outbound" to communicate with the hosted control plane.
| Outbound Port | Security | Purpose |
|---|---|---|
| 443/tcp | TLS with Mutual Auth | Connectivity to hosted control plane |
Controller IP¶
Customers that wish to lock down agent-to-control plane communications can optionally whitelist the IP addresses of the hosted control plane in their firewalls to ensure that outbound connectivity is only allowed to these IPs.
IP Addresses for Backend Services¶
The hosted control plane's backend services are deployed in a highly available configuration and uses a pool of load balanced IP addresses. Please add these to your firewall's whitelist if you are configuring IP based policies.
| IP Address | IP Address |
|---|---|
| IP Address 1 | 34.213.28.26 |
| IP Address 2 | 52.10.172.195 |
| IP Address 3 | 35.80.255.48 |
IP Addresses for Frontend Services¶
The hosted control plane's frontend services (i.e. user facing resources) are deployed in a highly available configuration and uses a pool of load balanced IP addresses. Please add these to your firewall's whitelist to ensure that end users using web browsers etc are able to access these IP addresses.
| IP Address | IP Address |
|---|---|
| IP Address 1 | 34.209.101.215 |
| IP Address 2 | 44.224.29.11 |
| IP Address 3 | 44.238.5.175 |
| IP Address 4 | 52.10.173.46 |
Domains¶
The whitelabeled environment assigned for customers uses the following URL pattern. Every customer is provided a unique domain with the "customer name" in the domain. Please ensure your users and systems in your datacenter are able to access these domains.
- User Console: https://rafay-poc-customername.gpupaas.ai
- Ops Console : https://ops-rafay-poc-customername.gpupaas.ai
For example, for ACME, URLs to whitelist would be
- User Console: https://rafay-poc-acme.gpupaas.ai
- Ops Console : https://ops-rafay-poc-acme.gpupaas.ai
Product Documentation¶
For user access to product documentation, please whitelist the URL "https://docs.rafay.co"
Container Registry¶
Rafay Registry¶
Container images for orchestration will be dynamically downloaded from a Rafay container registry in the hosted control plane. Please whitelist the following domains to allow for downloads of container images from ".rafay-edge.net" and ".dev.rafay-edge.net".
*.rafay-edge.net*.dev.rafay-edge.net
NVIDIA Registry¶
NVIDIA hosts software for the GPU Operator, drivers, Dynamo and NIM in their registry. Please whitelist the following.
*.nvcr.io*.nvidia.com
Info
If are using RedHat's Satellite, please ensure that Satellite is configured to access the above domains, download the required packages and make them available internally via a repo name cuda-rhelx-arch.
Ingress Nginx¶
Web based applications such as Jupyter Notebooks, Inference etc are generally exposed to users via an Ingress Controller. A popular option is Ingress Ngnix. Please whitelist registry.k8s.io to allow downloads of container images for the Ingress Controller.
Jupyter Notebooks, vLLM, Developer Pods¶
If you intend to use the Jupyter Notebook SKU or the vLLM Inference SKU or the Developer Pod OS SKUs, please whitelist the following.
*.docker.io*.ghcr.io*.quay.io*.docker.pkg.dev*
Hugging Face Model Registry¶
If you plan to use Rafay Token Factory or Inference service to deploy and operate LLMs with models downloaded from Hugging Face, please whitelist the following.
*.huggingface.co*.hf.cocas-bridge.xethub.hf.co
The last domain is Hugging Face's Xet data storage layer. It acts as a Content Addressable Storage (CAS) bridge endpoint to route, chunk, or fetch large model and dataset files during uploads and downloads.
Info
If you are using infrastructure on GCP, please whitelist us.gcp.cdn.hf.co as well.