Infrastructure ¶

The infrastructure is configured as code via Terraform, for various reasons.

Architecture ¶

System interconnections ¶

flowchart LR
    benefits[Benefits application]
    style benefits stroke-width:5px
    recaptcha[Google reCAPTCHA]
    rider((User's browser))
    idg[Identity Gateway]
    elig_server[Eligibility Server]
    ac_data[(Agency Card data)]
    cookies[(Cookies)]

    benefits -->|Errors| sentry
    elig_server -->|Errors| sentry

    rider --> benefits
    rider -->|Credentials and identity proofing| Login.gov
    rider --> recaptcha
    rider -->|Payment card info| Littlepay
    rider -->|Events| Amplitude
    rider -->|Session| cookies

    benefits --> idg
    benefits <--> recaptcha
    benefits -->|Events| Amplitude
    benefits -->|Group enrollment| Littlepay
    benefits --> elig_server

    subgraph "Agency Cards (e.g. MST Courtesy Cards)"
    elig_server --> ac_data
    end

    idg --> Login.gov
    Login.gov -->|User attributes| idg
    idg -->|User attributes| benefits

Benefits application ¶

flowchart LR
    internet[Public internet]
    frontdoor[Front Door]
    django[Django application]
    interconnections[Other system interconnections]

    internet --> Cloudflare
    Cloudflare --> frontdoor
    django <--> interconnections

    subgraph Azure
        frontdoor --> NGINX

        subgraph App Service
            subgraph Custom container
                direction TB
                NGINX --> django
            end
        end
    end

Front Door also includes the Web Application Firewall (WAF) and handles TLS termination. Front Door is managed by the DevSecOps team.

Ownership ¶

The following things in Azure are managed by the California Department of Technology (CDT)’s DevSecOps (OET) team:

Subcriptions
Resource Groups
Networking
Front Door
- Web Application Firewall (WAF)
- Distributed denial-of-service (DDoS) protection
IAM
Service connections

Environments ¶

Within the CDT Digital CA directory (how to switch), there are two Subscriptions, with Resource Groups under each. Each environment corresponds to a single Resource Group, Terraform Workspace, and branch.

Environment	Subscription	Resource Group	Workspace	Branch
Dev	`CDT/ODI Development`	`RG-CDT-PUB-VIP-CALITP-D-001`	`dev`	`dev`
Test	`CDT/ODI Development`	`RG-CDT-PUB-VIP-CALITP-T-001`	`test`	`test`
Prod	`CDT/ODI Production`	`RG-CDT-PUB-VIP-CALITP-P-001`	`default`	`prod`

All resources in these Resource Groups should be reflected in Terraform in this repository. The exceptions are:

Secrets, such as values under Key Vault. prevent_destroy is used on these Resources.
Things managed by DevSecOps

You’ll see these referenced in Terraform as data sources.

For browsing the Azure portal, you can switch your Default subscription filter.

Making changes ¶

Terraform is plan‘d when code is pushed to any branch on GitHub, then apply‘d when merged to dev. While other automation for this project is done through GitHub Actions, we use an Azure Pipeline (above) for a couple of reasons:

Easier authentication with the Azure API using a service connnection
Log output is hidden, avoiding accidentally leaking secrets

Local development ¶

Get access to the Azure account through the DevSecOps team.
Install dependencies:
- Azure CLI
- Terraform - see exact version in deploy.yml
Authenticate using the Azure CLI.
```
az login
```
Outside the dev container, navigate to the terraform/ directory.
Initialize Terraform. You can also use this script later to switch between environments.
```
./init.sh <env>
```
Create a local terraform.tfvars file (ignored by git) from the sample; fill in the *_OBJECT_ID variables with values from the Azure Pipeline definition.
Make changes to Terraform files.
Preview the changes, as necessary.
```
terraform plan
```
Submit the changes via pull request.

For Azure resources, you need to ignore changes to tags, since they are automatically created by Azure Policy.

lifecycle {
  ignore_changes = [tags]
}

Naming conventions ¶

The DevSecOps team sets the following naming convention for Resources:

<<Resource Type>>-<<Department>>-<<Public/Private>>-<<Project Category>>-<<Project Name>>-<<Region>><<OS Type>>-<<Environment>>-<<Sequence Number>>

Sample Names ¶

RG-CDT-PUB-VIP-BNSCN-E-D-001
ASP-CDT-PUB-VIP-BNSCN-EL-P-001
AS-CDT-PUB-VIP-BNSCN-EL-D-001

Resource Types ¶

Use the following shorthand for conveying the Resource Type as part of the Resource Name:

Resource	Convention
App Service	`AS`
App Service Plan	`ASP`
Virtual Network	`VNET`
Resource Group	`RG`
Virtual Machine	`VM`
Database	`DB`
Subnet	`SNET`
Front Door	`FD`

Azure environment setup ¶

The following steps are required to set up the environment, with linked issues to automate them:

terraform apply
Set up Slack notifications by creating a Slack email for the #benefits-notify channel, then setting it as a Secret in the Key Vault named slack-benefits-notify-email
Set required App Service configuration and configuration by setting values in Key Vault (the mapping is defined in app_service.tf)
Set up webhook from GitHub to App Service Deployment Center for the Packages event

This is not a complete step-by-step guide; more a list of things to remember. This may be useful as part of incident response.