Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: add ADR for RAG Evaluations Framework #842

Merged
merged 25 commits into from
Oct 4, 2024
Merged

docs: add ADR for RAG Evaluations Framework #842

merged 25 commits into from
Oct 4, 2024

Conversation

jalling97
Copy link
Contributor

@jalling97 jalling97 commented Jul 26, 2024 •
edited by justinthelaw
Loading

  • Adds the ADR for the overall RAG Evaluations Framework
  • This will be added to for each component of the RAG Evals MVP Epic and closed when all pieces are done and all decisions have been documented in the ADR.

@jalling97 jalling97 linked an issue Jul 26, 2024 that may be closed by this pull request
ADR: RAG Evaluations Framework #823
Closed
@netlify Netlify
Copy link

netlify bot commented Jul 26, 2024 •
edited
Loading

✅ Deploy Preview for leapfrogai-docs canceled.

Name Link
🔨 Latest commit 04a482f
🔍 Latest deploy log https://app.netlify.com/sites/leapfrogai-docs/deploys/66ff126d5a9ad6000840e81f

@jalling97 jalling97 self-assigned this Jul 26, 2024
@jalling97 jalling97 added the ADR 🧐 Architecture Decision Record label Jul 26, 2024
@jalling97 jalling97 changed the title (ADR) RAG Evaluations Framework adr: RAG Evaluations Framework Aug 27, 2024
@jalling97 jalling97 changed the title adr: RAG Evaluations Framework docs: RAG Evaluations Framework Aug 27, 2024
@jalling97 jalling97 added the documentation Improvements or additions to documentation label Aug 27, 2024
@jalling97 jalling97 changed the title docs: RAG Evaluations Framework docs: add RAG Evaluations Framework Aug 27, 2024
@jalling97 jalling97 changed the title docs: add RAG Evaluations Framework docs: add ADR for RAG Evaluations Framework Aug 27, 2024
barronstone
barronstone reviewed Sep 10, 2024
View reviewed changes
adr/0007-rag-eval-framework.md Show resolved Hide resolved
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
@jalling97 jalling97 marked this pull request as ready for review September 30, 2024 21:36
@jalling97 jalling97 requested a review from a team as a code owner September 30, 2024 21:36
justinthelaw
justinthelaw reviewed Oct 1, 2024
View reviewed changes
Copy link
Contributor

@justinthelaw justinthelaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks excellent so far! Some minor comments and nits.

adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
adr/0007-rag-eval-framework.md Show resolved Hide resolved
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
justinthelaw
justinthelaw previously approved these changes Oct 3, 2024
View reviewed changes
Copy link
Contributor

@justinthelaw justinthelaw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love it!

gphorvath
gphorvath previously approved these changes Oct 3, 2024
View reviewed changes
CollectiveUnicorn
CollectiveUnicorn requested changes Oct 3, 2024
View reviewed changes
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
CollectiveUnicorn
CollectiveUnicorn reviewed Oct 3, 2024
View reviewed changes
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
adr/0007-rag-eval-framework.md Outdated Show resolved Hide resolved
@jalling97 jalling97 dismissed stale reviews from gphorvath and justinthelaw via c654c25 October 3, 2024 20:43
gregclark-defenseunicorns
gregclark-defenseunicorns reviewed Oct 3, 2024
View reviewed changes
Copy link
Contributor

@gregclark-defenseunicorns gregclark-defenseunicorns left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! The only thing I want to point out is the about the model cards living in UDS runtime. That is definitely an assumption and needs validation from the UDS teams in general so maybe it is worth specifying that so it is not misunderstood.

@jalling97 jalling97 merged commit e3baa4d into main Oct 4, 2024
18 checks passed
@jalling97 jalling97 deleted the 823-adr-rag-evaluations-framework branch October 4, 2024 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@gregclark-defenseunicorns gregclark-defenseunicorns gregclark-defenseunicorns left review comments

@barronstone barronstone barronstone left review comments

@CollectiveUnicorn CollectiveUnicorn CollectiveUnicorn approved these changes

@justinthelaw justinthelaw justinthelaw approved these changes

@gphorvath gphorvath gphorvath left review comments

Assignees

@jalling97 jalling97

Labels
ADR 🧐 Architecture Decision Record documentation Improvements or additions to documentation
Projects
None yet
Milestone
Current - RAG UX Enhancements | Model...
Development

Successfully merging this pull request may close these issues.

ADR: RAG Evaluations Framework
6 participants
@jalling97 @barronstone @gphorvath @justinthelaw @CollectiveUnicorn @gregclark-defenseunicorns