Skip to content

Post-launch

Verify real user enrollments are starting to happen

Amplitude

Amplitude chart config with settings described in detail below

We consider a transit provider officially onboarded to Cal-ITP Benefits when the transit provider appears in our metrics. Specifically, the transit provider is onboarded when we see one or more complete enrollments for that transit provider in Amplitude.

Use this query to confirm:

  • Segment: All Users
  • Measured as: Event Totals
  • Events: Completed enrollment
    • User property enrollment_method: digital, in_person
    • Grouped by: transit_agency

You can also go directly to the existing Enrollments by transit provider chart.

Metabase

Amplitude currently stores only a year of historical data, so we archive all Cal-ITP data in Metabase. Thus, we also need to ensure metrics for the new transit provider are successfully piped from Amplitude to Metabase.

Use this query to confirm:

Metabase chart config with settings described in detail below

  • Data: Fct Benefits Events
  • Filter:
    • Event Type is returned enrollment
    • Event Properties Status is success
    • Event Time is [some date range that includes the dates you’re expecting to see data for the new transit provider]
  • Summarize: Count by Event Properties Transit Agency

You can also go directly to the existing Cal-ITP Benefits enrollments by transit provider 2025 chart.

Once the transit provider is live in production, there are some cleanup steps to take.

These items can all be done in parallel, and can also be done in parallel with the analytics verification described above.

Update transit provider configuration in test environment

The transit provider’s configuration in the test environment should be updated to change the production values back to the QA values for its steady state going forward.

Because Switchio’s acceptance environment is entirely distinct and isolated from production, nothing needs to be reverted.