Infrastructure ¶
The infrastructure is configured as code via Terraform, for various reasons.
Architecture ¶
System interconnections ¶
flowchart LR
benefits[Benefits application]
style benefits stroke-width:5px
recaptcha[Google reCAPTCHA]
rider((User's browser))
idg[Identity Gateway]
elig_server[Eligibility Server]
ac_data[(Agency Card data)]
cookies[(Cookies)]
benefits -->|Errors| sentry
elig_server -->|Errors| sentry
rider --> benefits
rider -->|Credentials and identity proofing| Login.gov
rider --> recaptcha
rider -->|Payment card info| Littlepay
rider -->|Events| Amplitude
rider -->|Session| cookies
benefits --> idg
benefits <--> recaptcha
benefits -->|Events| Amplitude
benefits -->|Group enrollment| Littlepay
benefits --> elig_server
subgraph "Agency Cards (e.g. MST Courtesy Cards)"
elig_server --> ac_data
end
idg --> Login.gov
Login.gov -->|User attributes| idg
idg -->|User attributes| benefits
Benefits application ¶
flowchart LR
internet[Public internet]
frontdoor[Front Door]
django[Django application]
interconnections[Other system interconnections]
internet --> Cloudflare
Cloudflare --> frontdoor
django <--> interconnections
subgraph Azure
frontdoor --> NGINX
subgraph App Service
subgraph Custom container
direction TB
NGINX --> django
end
end
end
Front Door also includes the Web Application Firewall (WAF) and handles TLS termination. Front Door is managed by the DevSecOps team.
Ownership ¶
The following things in Azure are managed by the California Department of Technology (CDT)’s DevSecOps (OET) team:
- Subcriptions
- Resource Groups
- Networking
- Front Door
- Web Application Firewall (WAF)
- Distributed denial-of-service (DDoS) protection
- IAM
- Service connections
Environments ¶
Within the CDT Digital CA
directory (how to switch), there are two Subscriptions, with Resource Groups under each. Each environment corresponds to a single Resource Group, Terraform Workspace, and branch.
Environment | Subscription | Resource Group | Workspace | Branch |
---|---|---|---|---|
Dev | CDT/ODI Development |
RG-CDT-PUB-VIP-CALITP-D-001 |
dev |
dev |
Test | CDT/ODI Development |
RG-CDT-PUB-VIP-CALITP-T-001 |
test |
test |
Prod | CDT/ODI Production |
RG-CDT-PUB-VIP-CALITP-P-001 |
default |
prod |
All resources in these Resource Groups should be reflected in Terraform in this repository. The exceptions are:
- Secrets, such as values under Key Vault.
prevent_destroy
is used on these Resources. - Things managed by DevSecOps
You’ll see these referenced in Terraform as data sources.
For browsing the Azure portal, you can switch your Default subscription filter
.
Making changes ¶
Terraform is plan
‘d when code is pushed to any branch on GitHub, then apply
‘d when merged to dev
. While other automation for this project is done through GitHub Actions, we use an Azure Pipeline (above) for a couple of reasons:
- Easier authentication with the Azure API using a service connnection
- Log output is hidden, avoiding accidentally leaking secrets
Local development ¶
- Get access to the Azure account through the DevSecOps team.
-
Install dependencies:
-
Terraform - see exact version in
deploy.yml
az login
- Outside the dev container, navigate to the
terraform/
directory. - Initialize Terraform. You can also use this script later to switch between environments.
./init.sh <env>
- Make changes to Terraform files.
- Preview the changes, as necessary.
terraform plan
For Azure resources, you need to ignore changes to tags, since they are automatically created by Azure Policy.
lifecycle {
ignore_changes = [tags]
}
Naming conventions ¶
The DevSecOps team sets the following naming convention for Resources:
<<Resource Type>>-<<Department>>-<<Public/Private>>-<<Project Category>>-<<Project Name>>-<<Region>><<OS Type>>-<<Environment>>-<<Sequence Number>>
Sample Names ¶
RG-CDT-PUB-VIP-BNSCN-E-D-001
ASP-CDT-PUB-VIP-BNSCN-EL-P-001
AS-CDT-PUB-VIP-BNSCN-EL-D-001
Resource Types ¶
Use the following shorthand for conveying the Resource Type as part of the Resource Name:
Resource | Convention |
---|---|
App Service | AS |
App Service Plan | ASP |
Virtual Network | VNET |
Resource Group | RG |
Virtual Machine | VM |
Database | DB |
Subnet | SNET |
Front Door | FD |
Azure environment setup ¶
The following steps are required to set up the environment, with linked issues to automate them:
terraform apply
- Set up Slack notifications by creating a Slack email for the #benefits-notify channel, then setting it as a Secret in the Key Vault named
slack-benefits-notify-email
- Set required App Service configuration and configuration by setting values in Key Vault (the mapping is defined in app_service.tf)
- Set up webhook from GitHub to App Service Deployment Center for the
Packages
event
This is not a complete step-by-step guide; more a list of things to remember. This may be useful as part of incident response.