adding better '<fields>' info with schema parser

[LeoGrosjean]

• Adding support to type
  "<value|type=string>"
• Adding support to description
  "<value|type=string|description=a description>"
• Adding support to Enum
  "<["value_1", ..., "value_n"]|type=string>"
• Adding support to List[Any]
  ["<value|type=string>"]
• Adding support to Format (replace type, tbd if not a good idea)
  ["<value|format=any>"]
• Adding fix to nested Object

Test prompt has been updated
Many test runned with openai models, all ending by being able to instanciate custom model

prompt = prompt_func(NestedModel)
NestedModel(**json.load(model(prompt)))

TODO :

• Add support to const

  class TemplateValueTask(str, Enum):
      task = "Tâche"
  
  #'TemplateValueTask': {'const': 'Tâche',
  # 'title': 'TemplateValueTask',
  # 'type': 'string'}

  class TemplateValueTask(BaseModel):
    task: Literal["Tâche"]
  
  # {'properties': {'task': {'const': 'Tâche', 'title': 'Task'}},
  # 'required': ['task'],
  # 'title': 'TemplateValueTask',
  # 'type': 'object'}

[LeoGrosjean]

broken i monkey patch it (todo added)
use of try except a bit ugly, but do the job until refacto

[LeoGrosjean]

For long schema, openai return max 1000 chr => easy to ask chatgpt to finish the json

I will make a way to allow user to tell how many max api call could be made to fulfill the json response

late but i can detail more

Do you want another pull request for it ?

[brandonwillard]

@LeoGrosjean, thanks for the contribution! We'll follow up shortly with some questions and review comments.

[LeoGrosjean]

With xxlarge schema, the output could be text and not json

occured 2 times with a willing confused prompt (bad field's description + text unrelated to schema)
> with gpt-3.5-turbo

could be controlled if outputs[0] not in ["{", "["]

OUT OF CONTEXT, to be discuss elsewhere, might be due to syntax issue

[LeoGrosjean]

I'm currently reading json schema conventions that pydantic is based on

I'll find a way to code more properly how we will parse schema from model_dump_json method

I'm currently rushing and monkey patching it for personal needs; but its working decently
same walking logic could be used elsewhere in outlines

Will update here a mermaid graph with the workflow I figure out

flowchart TD
    A[Model] -->|model_dump_json| B(raw schema)
    value[value]
    type[type]
    format[format]
    description[description]

Loading

[rlouf]

Any updates on this?

[LeoGrosjean]

holy kid has arrived, got code on my other computer, need to take time to push it !

but didnt take time to update the mermaid graph

will do it this week between two diapers :)

[LeoGrosjean]

• feature ready
• code quality (might need a review/peer if anyone available)
• writing tests for conditions features
• real life usecase with any transformer