Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add base moderation to Providers #513

Merged
merged 8 commits into from
Jan 10, 2025
Merged

Add base moderation to Providers #513

merged 8 commits into from
Jan 10, 2025

Conversation

zakiali
Copy link
Collaborator

@zakiali zakiali commented Dec 20, 2024

Adding in moderation for user inputs.

  • A Moderation trait is added to the base Provider struct along
  • Update the base provider complete method to make a concurrent request to both the moderation and LLM endpoint. If moderation fails, we error out and abort the LLM completion if the request is still in flight. If moderation passes, we wait for the LLM response and continue as normal
  • Only moderates on the latest User message
  • Implemented moderation for the OpenAI provider using the openai's moderation endpoint, as an example

@zakiali zakiali marked this pull request as ready for review December 20, 2024 22:26
@zakiali zakiali force-pushed the zaki/moderation branch 7 times, most recently from 2199bad to e89e10d Compare January 3, 2025 19:00

let response_json: serde_json::Value = response.json().await?;

let flagged = response_json["results"][0]["flagged"]
Copy link
Collaborator

@ahau-square ahau-square Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we check response status before parsing? I think we use handle_response elsewhere


// Get the content to moderate
let content = latest_user_msg.content.first().unwrap().as_text().unwrap();
println!("Content to moderate: {}", content);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove

@zakiali zakiali force-pushed the zaki/moderation branch 8 times, most recently from 7ded65b to 92fd00a Compare January 10, 2025 01:28
Copy link

Desktop App for this PR

The following build is available for testing:

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

Copy link

Desktop App for this PR

The following build is available for testing:

The app is signed and notarized for macOS. After downloading, unzip the file and drag the Goose.app to your Applications folder.

This link is provided by nightly.link and will work even if you're not logged into GitHub.

@zakiali zakiali merged commit 7b827d0 into v1.0 Jan 10, 2025
6 checks passed
michaelneale added a commit that referenced this pull request Jan 14, 2025
* v1.0:
  feat: static settings page (#570)
  styles: adding arcade styles and cash sans (#581)
  feat: quick spinner while loading tokenizers (#573)
  Add timeout middleware for clients (#572)
  chore: remove unused old setup for CLI  (#574)
  feat: env and secrets configuration for mcp server (#565)
  Add Databricks moderation (#540)
  feat: add pagination support for tools/list and resources/list (#566)
  Add resource capabilties to MCP servers that use it (#576)
  Add goose versions to the UI (#526)
  fix: Set stdin to null in shell/bash tools (#568)
  Add base moderation to Providers (#513)
  feat: set process_group(0) on stdio systems to avoid ctrl-c handling (#567)
  feat: read only active resources in the agent loop (#560)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants