Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add Langfuse Integration and Fix Audio Processing #355

Merged
merged 40 commits into from
Jan 18, 2025
Merged

Conversation

MrOrz
Copy link
Member

@MrOrz MrOrz commented Jan 1, 2025

This PR adds Langfuse integration and includes several fixes for audio processing and OpenAI-related functionality.

Main Changes

Langfuse Integration

  • Initial Langfuse setup. However, this requires us to upgrade OpenAI SDK
  • Use AI response ID as trace ID, so that we can score these traces easily in the future

OpenAI & Testing Updates

Upgraded OpenAI SDK, which broke a few things and need repairs.

  • Fixed OpenAI SDK method typo
  • Updated OpenAI mock implementations to reflect API changes
  • Fixed circular reference and formatting in CreateAIReply.js mocks
  • Temporarily skipped audio transcription test due to fluent-ffmpeg issue

Build & Configuration

  • Enabled experimental-vm-modules for Jest compatibility with Langfuse SDK
  • Fixed Langfuse import issues

The changes primarily focus on improving audio processing reliability and integrating Langfuse while maintaining test coverage and fixing various related issues.

Example Langfuse traces

@johnson-liang johnson-liang changed the title Record AI transcripts and replies to Langfuse feat: Add Langfuse Integration and Fix Audio Processing Jan 15, 2025
@coveralls
Copy link

coveralls commented Jan 15, 2025

Coverage Status

coverage: 82.699% (-0.7%) from 83.364%
when pulling 61a3497 on langfuse
into 5dc1e5e on master.

@MrOrz
Copy link
Member Author

MrOrz commented Jan 16, 2025

After deploying to staging and testing it using videos, I've got some errors. Staging bot is not working now. Going back to draft state.

Issue #1: output stream closed

Similar to fluent-ffmpeg/node-fluent-ffmpeg#1135 , somehow Whisper will close connection early and triggers error on fluent-ffmpeg.
image

It seems that the logic still works though, may consider ignoring the issue.

Issue #2: AI reply bug

Seems that we should fix CreateAIReply. The LLM call is mocked so that the change to the interface is not captured in the unit test.
image

@MrOrz MrOrz marked this pull request as draft January 16, 2025 06:35
@MrOrz MrOrz marked this pull request as ready for review January 16, 2025 19:29
@MrOrz MrOrz merged commit 49199d5 into master Jan 18, 2025
5 of 7 checks passed
@MrOrz MrOrz deleted the langfuse branch January 20, 2025 05:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants