Filling out household information with the new AI Explanation Feature

[BenGrossBG]

One thing I have noticed since we introduced the "Explain with AI" feature are a few odd results in the generation when I only fill out the minimum household information.

For example, when you ask it to explain WIC benefit for this household in Connecticut, filling out only the default fields, it says:

In this simulation, a household of four individuals was analyzed for WIC eligibility. The result shows that three members are classified as 'NONE' for WIC, while one is classified as 'CHILD'.

The main factors influencing this outcome are:
Age: The household members are 40, 40, 6, and 3 years old.
Gender: None of the individuals are female.
Parental status: No one is identified as a parent or pregnant.
Breastfeeding status: No one is breastfeeding.

Of course, however, most people who fill out a household like this are describing a related family unit. This is fixable by the user, but it's a bit tricky to figure which variables you need (is female and # of own dependents in the household who are your own children) to fix. Does anyone have ideas for preventing these confusing generations? Is it a problem we have to live with? Excited to hear your thoughts.

[anth-volk]

Loving Claude's recommended changes:

Changes that could affect this result include:

    If any adult were female and became pregnant or began breastfeeding
    If the 6-year-old were younger
    If income data were included and fell within eligible ranges

If only we could reverse aging...

But to get to your point, Ben, more broadly, I think putting guardrails up for how Claude responds to things has been an issue in general. Separately, in our society-wide AI explainer, when we asked it to "explain like I'm 5," Claude would actually refuse to do so, saying that it wouldn't be willing to dumb down tax policy. We need to investigate placing firmer restrictions, perhaps through system messages?