Skip to content

Implement SSE streaming for Vue 3, Backend and Flutter

Backend Streaming (SSE) for Vue Frontend

Context

The ChatQnA microservice already supports OpenAI-compatible SSE streaming. However, the Vue 3 frontend (ChatBotComponent.vue) does not consume the streaming endpoint — messages appear only after the full LLM response completes. This is the single biggest UX gap and a prerequisite for agentic workflows which need real-time status updates.

Scope

Frontend (Vue 3)

  • Modify ChatBotComponent.vue to consume the SSE streaming endpoint (/v1/chat/completions with stream: true)
  • Display tokens incrementally as they arrive (typewriter effect)
  • Handle streaming errors, timeouts, and partial message recovery
  • Ensure streaming works with the existing Vuex store and message model

Backend (Node.js)

  • Verify the backend BFF proxy correctly streams SSE responses from ChatQnA to the frontend
  • Add proper SSE headers (Content-Type: text/event-stream, Cache-Control: no-cache, Connection: keep-alive)
  • Handle Kong proxy SSE passthrough (verify nginx WebSocket/streaming config)
  • Add streaming support to the Flutter mobile app's chat interface

Mobile (Flutter)

  • Update the chat service to handle SSE streams
  • Implement incremental token display in the Flutter chat UI

Acceptance Criteria

  • Vue frontend displays LLM responses token-by-token via SSE
  • Backend correctly proxies SSE from ChatQnA without buffering
  • Kong/nginx pass SSE through without buffering or timing out
  • Flutter mobile app supports streaming responses
  • Error handling for stream interruptions (network drop, timeout, service down)
  • Streaming works with and without translation enabled

Dependencies

  • Requires microservice auth (#500 (closed)) to be resolved first (secure endpoints before streaming)

Labels

enhancement, frontend, backend, mobile