fishies-order-app

Fishies Order App

Three ways for a restaurant to order fish from a distributor running on the Aquarius ERP:

1. Mobile / web app — browse catalog, build a cart, place orders, chat with an AI agent (Gemini on Vertex AI) by text or voice.
2. AI chat with voice in the app — record on-device, transcribe via Google Speech-to-Text, the agent uses tool calls to drive the Aquarius API.
3. Phone-call ordering — call a Twilio number, talk to “Alice” (OpenAI Realtime), she finds the customer, builds the order, reads it back, and writes it to Aquarius.

fishies-order-app/
├── app/                    # Expo (React Native) — iOS / Android / web
├── server/                 # Node/Express — JSON API + Twilio voice pipeline
│   └── src/
│       ├── app.js          # createApp() factory (testable)
│       ├── index.js        # boots HTTP server + Twilio media-stream WebSocket bridge
│       ├── aquarius.js     # low-level Aquarius fetch (token-aware)
│       ├── chat.js         # Vertex Gemini + tool calls
│       ├── stt.js          # Google Speech-to-Text
│       ├── auth.js         # requireAuth — forwards Firebase ID token
│       ├── routes/twilio/  # /twilio/voice, /twilio/status, /api/twilio/*
│       ├── services/       # OpenAI realtime bridge, Twilio client, order tools, session store
│       └── data/           # seed catalog (used as fallback before live customer is matched)
├── Dockerfile              # Cloud Run image (server only)
├── .github/workflows/      # CI/CD: tests + deploy to Cloud Run
└── .env                    # Shared dev secrets (loaded by server; not committed)

Quick start

# 1. install deps
npm install                  # root (concurrently) — also hoists workspace deps
npm install --prefix app     # for app web/native bits if you skipped the workspace install

# 2. run both server + app
npm run dev
# server: http://localhost:8787
# app:    Expo dev server — scan QR with Expo Go, or press i/a/w

The server reads .env at the repo root via node --env-file=../.env. The Aquarius API token, GCP service-account path, OpenAI key, and Twilio creds all stay server-side. The mobile app only talks to EXPO_PUBLIC_API_BASE_URL, and the only client-side keys it gets are the (public-by-design) Firebase web config.

Features

• Catalog — live /api/v1/stock, filter by category, search.
• Cart — add lines, set qty/weight, place an order tied to a customer code.
• Orders — recent sales orders + line details.
• Chat with AI agent — text or voice. Tools: search_products, list_recent_orders, get_order, place_order. Voice in-app uses expo-av recording → /api/stt → Google Speech-to-Text → chat.
• Phone-call ordering — Twilio inbound webhook hits /twilio/voice, returns TwiML connecting the caller to /twilio/media-stream (WebSocket). The bridge proxies audio to OpenAI Realtime, which calls tools (findCustomer, searchProducts, addItemToOrder, confirmOrder, saveOrder) to build and persist the order in Aquarius.
• Email ordering — a Gmail users.watch() on the orders mailbox streams change notifications through Cloud Pub/Sub → a push to /gmail/push. The handler verifies the Pub/Sub OIDC token, replays Gmail history, resolves the recipient mailbox → tenant (unknown mailboxes are skipped, never default-routed) and the sender → customer, runs the agent to EXTRACT the order, and writes a received_orders row (channel email, idempotent on the RFC822 Message-ID). No outbound auto-reply — humans confirm. Re-arm the ~7-day watch with node scripts/gmail-register-watch.js on a cron. See GMAIL_* in .env.example.
• Auth — Firebase email/password gate on the app; ID token is forwarded server-side and on through to Aquarius.

Endpoints

Method	Path	Auth	Notes
GET	/api/health	none	Liveness
GET	/openapi.json, /docs	none	Swagger UI
GET	/api/aquarius/*	Firebase	Allowlisted read proxy
POST	/api/aquarius/orders	Firebase	Create sales order + lines
POST	/api/chat	Firebase	Gemini + tool calls
POST	/api/stt	Firebase	Multipart audio → transcript
GET	/api/twilio/config	none	Webhook URL inspection
POST	/api/twilio/configure-number	none	Re-points Twilio number at this server
POST	/twilio/voice	(Twilio sig)	Returns TwiML
POST	/twilio/status	(Twilio sig)	Call lifecycle webhook
WS	/twilio/media-stream	n/a	Bidirectional audio with OpenAI Realtime
POST	/gmail/push	(Pub/Sub OIDC)	Inbound email order ingest (Gmail watch → Pub/Sub push)

Tests

npm --prefix server test

Covers: config defaults, port-fallback listener, Twilio voice TwiML rendering, body parser, number configuration, fuzzy matching, full order tool happy-path + repository persistence + error handling, and the Twilio media-stream control logic (filler-detection, audio interruption, hangup gating).

Deploy

Push to main → GitHub Actions builds the server Docker image and deploys it to Cloud Run as fish-ordering-assistant-api-run in europe-west2. After deploy it sets PUBLIC_BASE_URL to the Cloud Run URL, opens IAM to public invokers, then calls its own /api/twilio/configure-number to point Twilio’s voice + status webhooks at the new URL.

Required GitHub secrets:

• GOOGLE_APPLICATION_CREDENTIALS — JSON of a service account with Cloud Run, Cloud Build, Artifact Registry, Vertex AI, and Speech roles. The repo already targets wakeflow-aquarius.
• .env values are layered in via npx @wakeflow/secrets at deploy time, then converted to .env.yaml for gcloud run deploy --env-vars-file.

The Expo app builds independently — npx expo export -p web produces a static bundle you can host on any CDN, or use EAS for iOS/Android.