Skip to content

feat: Implement SSE for live log streaming and add retries for API calls

Edouard Legoupil requested to merge fix/knowledge-card-streaming-and-retries into main_dev

Created by: Edouard-Legoupil

This commit addresses two issues with the knowledge card generation feature:

  1. Live Log Streaming: The previous implementation used polling to update the generation progress. This has been replaced with a Server-Sent Events (SSE) stream for real-time updates.

    • The backend now uses a Redis pub/sub channel to broadcast progress updates.
    • The /knowledge-cards/{card_id}/status endpoint has been converted to a streaming endpoint.
    • The frontend now uses EventSource to listen for these events and update the UI in real-time.

  2. API Rate Limiting: The application was crashing due to rate limit errors from the Azure OpenAI API.

    • A retry mechanism has been added to the litellm.embedding call by setting max_retries=3. This will make the application more resilient to transient network errors and rate limiting.
  • Does my code meet the quality standards for releasing packages?
  • Does the reviewer have all the information to validate the features/issues without too much research?
  • Does the customer who will validate the associated tickets have the information to do so without wasting time?

Issues to validate to close :

  • issue #

Processed issues to keep open or in progress:

  • issue #

Checklist:

  • Does the package check go local?
  • Does the CI pass?
  • Are the added / fixed features documented, tested?
  • Are the added features / solved problems briefly presented in the PR message?
  • Are the changes related to tickets / issues that I have listed in the commits and in the PR itself?
  • Are the tickets in "review" mode in the Project Tracking Board?
  • Does each ticket, if it is to be closed after acceptance of the PR, contain a comment that tells how to validate it?

Merge request reports