Skip to content

Commit

Permalink
Add experimental-warning to CFGGuide, update warning-linked docs
Browse files Browse the repository at this point in the history
  • Loading branch information
lapp0 authored and brandonwillard committed Jul 27, 2024
1 parent d843ec6 commit 019b100
Show file tree
Hide file tree
Showing 2 changed files with 13 additions and 2 deletions.
9 changes: 7 additions & 2 deletions docs/reference/cfg.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,10 +30,15 @@ print(sequence)
# (8-2)
```

!!! Note "Performance"
###### Disclaimer

The implementation of grammar-structured generation in Outlines is very naive. This does not reflect the performance of [.txt](https://dottxt.co)'s product, where we made grammar-structured generation as fast as regex-structured generation.
!!! Note "Experimental"

Outlines current **community-contributed** implementation of CFG-structured generation is experimental. This does not reflect the performance of [.txt](https://dottxt.co)'s product, where we have optimized grammar-structured generation to be as fast as regex-structured generation. Additionally, it does not fully align with the approach described in our [technical report](https://arxiv.org/pdf/2307.09702), aside from its use of incremental/partial parsing. This feature is still a work in progress, requiring performance enhancements and bug fixes for an ideal implementation. For more details, please see our [grammar-related open issues on GitHub](https://github.com/outlines-dev/outlines/issues?q=is%3Aissue+is%3Aopen+label%3Agrammar).

!!! Note "Greedy"

To mitigate performance issues, CFG-structured generation will use rejection sampling and iterate over the candidate tokens highest logit first,, completing once a single valid token ID is selected. This is effectively greedy generation.

## Ready-to-use grammars

Expand Down
6 changes: 6 additions & 0 deletions outlines/fsm/guide.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import collections
import copy
import warnings
from dataclasses import dataclass
from typing import (
TYPE_CHECKING,
Expand Down Expand Up @@ -319,6 +320,11 @@ def __init__(self, cfg_string: str, tokenizer):
"""
Construct the PartialLark parser and set the empty initial_state (PartialParserState)
"""
warnings.warn(
"Outlines' public *community-contributed* CFG structured generation is experimental. "
"Please review https://outlines-dev.github.io/outlines/reference/cfg#disclaimer"
)

self.cfg_string = cfg_string
self.tokenizer = tokenizer
self.eos_token_id = self.tokenizer.eos_token_id
Expand Down

0 comments on commit 019b100

Please sign in to comment.