Apply for Hawaii SNAP / Financial Assistance (Voice + Document-Assisted)

Purpose

Guide a single user, conversationally, through the State of Hawaii Department of Human Services (DHS) online application for SNAP (food) and/or Financial (money) Assistance, hosted at https://pais-benefits.dhs.hawaii.gov/. The skill is an accessibility/assistance layer on top of the official browser form:

Text-to-speech (TTS): read each form question and its help text aloud to the user, one step at a time.
Whisper (speech-to-text): capture the user's spoken answer, normalize it (dates → mm/dd/yyyy, SSN → ###-##-####, phone → 808-555-5555 shape), confirm it back by voice, then type it into the field.
Document OCR/scan: when the user offers a photo or scan of a document (state ID, SSN card, pay stub, lease, utility bill, immigration card), extract the relevant fields and pre-fill the form, always reading extracted values back for confirmation before committing them.

This is a write flow (it fills a live government application), but it is submission-gated: the agent must never sign, certify, or submit the application. The final Certification / signature step is always performed by the user after they review the whole application. Reaching the Certification step and stopping is the success terminal state for the agent.

When to Use

A user who cannot comfortably type a long government form (vision impairment, motor impairment, low digital literacy, limited English) wants to apply for Hawaii SNAP/financial aid by talking.
A user has a stack of documents (ID, pay stubs, lease) and wants an agent to read the numbers off them into the correct fields.
Resuming a partially completed application (portal "FINISH MY APPLICATION") or attaching documents to a submitted one (portal "UPLOAD DOCUMENTS").
Not for renewals-only flows beyond starting them — renewals have their own portal button ("RENEW MY BENEFITS") and restricted hours (see gotchas).

Workflow

The application lives on pais-benefits.dhs.hawaii.gov (a server-rendered, multi-step IBM WebSphere form), not on the pais.dhs.hawaii.gov informational SPA. pais.dhs.hawaii.gov/PAIS/#!/how-to-apply just links out to the benefits portal with a "Click Here to Apply for Benefits" button. Go straight to the portal.

Session setup: a plain Browserbase session works — no stealth needed. The pre-run probe reported no anti-bot, and a bare session walked the entire flow cleanly.

sid=$(browse cloud sessions create --keep-alive \
  | node -e "let s='';process.stdin.on('data',c=>s+=c).on('end',()=>process.stdout.write(JSON.parse(s).id))")
export BROWSE_SESSION="$sid"
browse open "https://pais-benefits.dhs.hawaii.gov/" --remote --session "$sid"

Confirm eligibility and intent aloud. TTS: "You must be a resident of the State of Hawaiʻi. Are you applying for food benefits (SNAP), money benefits, or both?" Record the answer — it drives the first checkbox.
Enter the application. Click the APPLY NOW button ("Start a new application for SNAP or financial assistance"). (Other portal buttons: FINISH MY APPLICATION, UPLOAD DOCUMENTS, VIEW MY ISSUED BENEFITS, RENEW MY BENEFITS.) The app opens at /unsecure/getLEPPage — a Bilingual and Sign Interpreter Services info screen (a static image). Read the gist aloud, then click NEXT.
Work the Registration section (/unsecure/getRegistrationPage), which is a sequence of small sub-steps sharing one URL. For each: TTS the question, capture via Whisper, confirm, fill, click NEXT:
- Benefits type — checkboxes Money (Financial) and/or SNAP (Food).
- Interpreter Services — "Do you need an interpreter?" No / Yes.
- Primary Language — free-text "primary language spoken in your home" + a radio for English proficiency (5 levels from "Speaks well, adequate reading and writing" to "Does not speak or understand English").
- Applicant Information — First Name *, Middle Name, Last Name *, Phone Number, Gender (select: Male/Female), Social Security Number, Date of Birth, Email. This person is the primary contact even if not personally applying.
- Address — Home Address *, Apt/Space No., City *, State * (select — choose HI), plus ZIP/county. "If you have no home address, enter a mailing address where you can receive mail."
Walk the 15 main-application steps shown in the left sidebar, in order, same TTS→Whisper→confirm→fill→NEXT rhythm: Applicant Registration → Applicant Information → Household → Citizenship → Prior Benefits → Able Bodied Adults Without Dependents → Assets → School → Earned Income → Unearned Income → Expenses → Authorized Representative → Voter Registration → Document Upload → Certification. Use the progress bar ("% Complete") and the breadcrumb to tell the user where they are.
Document OCR pre-fill. Whenever the user supplies a document image, OCR it and map fields to the current step (examples below), read each extracted value back by voice, and only type it after the user confirms. At the Document Upload step, help the user attach supporting files (verification documents) rather than re-keying data.
Stop at Certification. Read the full summary / certification language aloud. Then hand control to the user to review, sign, and submit themselves. Do not click the sign/submit/certify control on the user's behalf. After submission the portal issues a Tracking # (shown in the header) and a receipt — relay it to the user.
Release the session when done: browse cloud sessions update "$sid" --status REQUEST_RELEASE.

Typical documents → fields map (for OCR)

State ID / driver's license → name, DOB, address.
Social Security card → SSN (confirm digit-by-digit by voice).
Pay stubs / employer letter → Earned Income (employer, gross amount, pay frequency).
Benefit award letters (SSI, SSA, unemployment, child support) → Unearned Income.
Lease / mortgage statement, utility bills, childcare/medical receipts → Expenses.
Immigration documents (I-551, EAD, etc.) → Citizenship/immigration status fields.

Site-Specific Gotchas

Two different hosts. Information + program descriptions live on pais.dhs.hawaii.gov/PAIS/ (an AngularJS #!/-hash SPA). The actual application lives on the separate host pais-benefits.dhs.hawaii.gov. Don't try to fill the form on the .gov/PAIS/ SPA — it only links out.
No anti-bot, no stealth needed. A bare Browserbase session (no --verified, no --proxies) completed LEP → Registration → Applicant Info → Address with zero blocks. Do not add stealth flags you don't need.
~15-minute idle session timeout. Every page shows a hidden "Your Session is About to Expire! Redirecting in NNNs" countdown (~887s observed) with Stay Connected / Logout buttons. Voice interaction is slow, so watch for and click Stay Connected before the countdown lapses, or the application resets and in-progress data is lost. Keep the user moving; don't leave a step idle for minutes.
Registration is multi-page under one URL. Benefits-type, interpreter, primary language, applicant info, and address are all served from /unsecure/getRegistrationPage and advance via server round-trips on NEXT — the URL does not change. Re-snapshot after every NEXT; stale a11y refs from the previous sub-step will not match.
Date of Birth is a native split date control, not one textbox. It renders as three spinbuttons (Month mm, Day dd, Year yyyy) plus a "Show date picker" button. Fill month/day/year separately (or focus and type mm, dd, yyyy in sequence); a single fill of mm/dd/yyyy into the group will not stick.
State select defaults to blank. For Hawaii residents choose HI. The select lists all US states/territories including FM, GU, AS, etc.
Primary-language English-proficiency answer is a radio, not free text. Only the "primary language spoken" box is free text.
Renewals have restricted hours. The RENEW MY BENEFITS button is "Available 6:30am–5:45pm HST Mon–Sat, except State holidays." A new application (APPLY NOW) is not similarly gated, but expect occasional maintenance windows.
Browser compatibility banner. The portal states it is "optimized for Google Chrome and Mozilla Firefox." Browserbase's Chromium is fine; dismiss the banner if present.
NEVER auto-submit. Verification is a legal attestation: "All of your answers must be complete and honest... you may incur substantial penalties, including fines, imprisonment..." The agent must not sign/certify. Fill, confirm, and hand off at Certification.
PII handling. This flow collects SSN, DOB, address, income. Read values back for confirmation but never log or transmit them anywhere except into the official form fields. Confirm the user consents before OCR-extracting a document.
No public API / deep link. The multi-step flow is server-stateful (WebSphere session cookie); there is no documented JSON endpoint to POST the application to. Confirmed: browser is the only surface. Don't waste time hunting for an API.

Expected Output

The skill's "output" is progress state + the data it placed into the form, not a scraped payload. Emit a structured status object the orchestrator can resume from:

{
  "status": "awaiting_user_certification",
  "portal": "https://pais-benefits.dhs.hawaii.gov/",
  "benefitsRequested": ["SNAP", "Financial"],
  "currentStep": "Certification",
  "stepsCompleted": [
    "Applicant Registration", "Applicant Information", "Household",
    "Citizenship", "Prior Benefits", "Able Bodied Adults Without Dependents",
    "Assets", "School", "Earned Income", "Unearned Income", "Expenses",
    "Authorized Representative", "Voter Registration", "Document Upload"
  ],
  "trackingNumber": null,
  "fieldsFilled": {
    "firstName": "confirmed",
    "lastName": "confirmed",
    "ssn": "confirmed-by-voice",
    "dateOfBirth": "mm/dd/yyyy",
    "address": { "state": "HI" }
  },
  "documentsOcred": ["state_id", "pay_stub_2025-09"],
  "submitted": false,
  "note": "Stopped at Certification. User must review, sign, and submit."
}

Other terminal shapes:

// Session expired mid-flow before the user could act
{ "status": "session_expired", "currentStep": "Earned Income", "submitted": false,
  "note": "15-min idle timeout elapsed; re-enter via FINISH MY APPLICATION." }

// Resumed an existing application
{ "status": "resumed", "entryButton": "FINISH MY APPLICATION", "submitted": false }

// User completed submission themselves after hand-off
{ "status": "submitted_by_user", "trackingNumber": "<shown in header receipt>", "submitted": true }