Example: Vertex AI Gen Eval service

[philschmid]

No description provided.

[alvarobartt]

Hi here @philschmid! Thanks for this PR! Just a couple of nits/questions:

• Should we use "we" over "you"? I recall we mentioned in the past to use e.g. "you need to run ..." instead of "we need to run ..."?

• The pipeline that automatically updates the examples listing is still WIP, and whilst the examples will be auto-generated within the docs; the tables would still need manual update (will try to push the CI for that later today as atm is just a local script)

• Should we maybe create another category besides training and inference for e.g. benchmarking or evaluation? Not needed within this PR just flagging that.

Besides those concerns, we're good to merge! 🤗

[alvarobartt]

P.S. Do you have a good or decent experience with any Jupyter Notebook reviewer tool? I believe that reviewing notebooks within this repository is not so odd, as most of the Vertex AI examples will run on Jupyter Notebooks; so maybe makes sense to add any if you had a good experience? I didn't TBH, but asking just in case you have any recommendation!

[philschmid]

Should we use "we" over "you"? I recall we mentioned in the past to use e.g. "you need to run ..." instead of "we need to run ..."?

I always use "we" in my examples

The pipeline that automatically updates the examples listing is still WIP, and whilst the examples will be auto-generated within the docs; the tables would still need manual update (will try to push the CI for that later today as atm is just a local script)

What do i have to update?

Should we maybe create another category besides training and inference for e.g. benchmarking or evaluation? Not needed within this PR just flagging that.

Haha good question. Evaluation might work?

any Jupyter Notebook reviewer tool?

never used one. Feel free to experiment

[alvarobartt]

I always use "we" in my examples

Then I'm happy to later align on comms and use "we" over "you" 👍🏻

What do i have to update?
Haha good question. Evaluation might work?

Assuming that we will create evaluation for the evaluation/benchmarking related examples; first of all you should include an initial Markdown block within the Jupyter Notebooks as:

---
title: Evaluate open LLMs with Vertex AI and Gemini
type: evaluation
---

Then update the following tables:

• Vertex AI -> Evaluation -> Add your example as \[title\]\(URL\) in https://github.com/huggingface/Google-Cloud-Containers/blob/main/docs/source/resources.mdx

• Create a new section for "Evaluation" and add it at the end of https://github.com/huggingface/Google-Cloud-Containers?tab=readme-ov-file#examples as ## Evaluation with a table including Service | Example | Title

• Same as above but in https://github.com/huggingface/Google-Cloud-Containers/tree/main/examples/vertex-ai without the Service column

P.S. Sorry for the extra manual work, I'll try to automate it properly so that those are automatically generated; as the script I had was not super consistent yet!

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[alvarobartt]

I think there was a mismatch between your main and the main branch, and this has been split in different files, so you should only add the table entry to the https://github.com/huggingface/Google-Cloud-Containers/blob/main/docs/source/resources.mdx file; while this should remain unchanged.

P.S. Sorry for the inconvenience!