[META] Usage Category based word definitions

[gregdan3]

Click here to read the discussion that prompted this issue

jan Kekan San
—
Today at 4:22 PM
ok i think i've arrived at the issue i have with broad definitions.

from my pov, including broad definitions (let's say, for argument's sake, 30% usage) is about the same as grabbing a random word from linku's Uncommon category and bringing it up to Core.

that category includes apeja, epiku, jasima, kiki, kokosila, linluwi, majuna, and powe

but i feel pretty confident in saying that you, and most everyone in this group, wouldn't want to include any of these in the core category (in like, a theoretical universe where we just hand picked words like that; we aren't doing that)

i don't see how adding a low usage definition to a word is different from elevating these words

---

so my questions:
is there something meaningfully different in elevating a low-usage definition of a core word vs elevating a non-core word?

if so, what is that difference? is it because we are weighting the resourcefulness of skilled speakers more heavily? is it because the definition made sense all along and should have been included in the first place? is it that we are weighting our own usage too heavily compared to considering the usage we have experienced? (all of these spiral into different, other questions)

if not (back to "meaningfully different") then why would we do that? it just makes our definitions less accurate to usage, which seems harmful to pedagogy
lipamanka
—
Today at 4:23 PM
hmmm what if we
what if we labeled usages using the same terminology
like "Uncommon:"
jan Kekan San
—
Today at 4:24 PM
i would actually be amazingly okay with that
lipamanka
—
Today at 4:24 PM
we could set it up so those only show up if you have the checkmark checked for that
jan Kekan San
—
Today at 4:24 PM
we'd have to figure out formatting and break api
lipamanka
—
Today at 4:24 PM
I am actually really happy with that ohmygod
if that could work that would be awesome
jan Kekan San
—
Today at 4:24 PM
well, there's one other problem
lipamanka
—
Today at 4:24 PM
we could even do a massive community survey at some point for those usages
hmm?
jan Kekan San
—
Today at 4:25 PM
that's what i was gonna say basically
the usage categories in linku now are driven by data

but the theoretical usage-split versions are not, because we don't have anything but the ku data to derive it from
lipamanka
—
Today at 4:26 PM
idea:
jan Kekan San
—
Today at 4:26 PM
if we pin them to the same checkbox, we're kinda lying
lipamanka
—
Today at 4:26 PM

we eyeball them for now
we plan on surveying the usage with data

we add a disclaimer saying that there's no data for them yet

I think the people who would care would read all of the notes about linku right
jan Kekan San
—
Today at 4:26 PM
yeah definitely
i'm gonna bring this to an issue
lipamanka
—
Today at 4:27 PM
awesome
okay i think we solved the issue and also made this project like 10x more descriptive
jan Kekan San
—
Today at 4:28 PM
well, and made more work for ourselves lmao
but it isn't accepted yet, more discussion first :P

We've seen a lot of discussion in #14 and #13 about what words should be included in a definition based on usage, with some of us siding with "broader" definitions and some siding with "narrower" definitions.

Summary

We should have usage categories in the definitions; for now, a "major" and "minor" group, which lets us divide more common/central meanings from less common/more distant meanings.

Proposal

We apply usage categories to definitions.
Right now this would be done informally, and we could survey later on to get more precise data in a way comparable to the ku surveys, but
If a word proposed for the definition suits the use of a majority of people, it goes to "core" or maybe a combined core+common (depends on what we groups we decide on); if a proposed word suits the use of a minority of people, but still a significant minority (we have to decide on a cutoff), it goes to "uncommon" or whatever the secondary category is. More divisions could be decided on later if it were necessary, since we would just have raw data.

Upsides

• More broad capture of usage without exclusions of potentially important or growing uses
• Less need to argue about a given usage, only where to put it; if it exists, we can determine its level of usage (see downsides) and categorize it based on that, allowing for a more complete record of usage without introducing less common meanings to the "core" definitions
• Dictionary becomes more useful to a wider range of people, since it can provide definitions reflective of marginal uses

Downsides

• We don't have data right now. We'll have to eyeball it.
• More work for us to fill out now (although honestly not that much more).
• UI/selectable option changes needed for ilo and lipu
• Increases translator work (potentially across more than one field)
• Not all of us will be familiar with a given marginal use of a word, leading to it being unrepresented (especially true if we change our work process)
• Tracking these marginal uses could manipulate the usage of words (this is more of a fact than a downside)

Considerations

• We have to decide if we're going full usage categories, or a smaller version of the usage categories that is more tenable (this was suggested in the Discord by kala Asi). I would say two categories, maybe "major" and "minor" or "common" and "marginal" or something to that effect, to distinguish the two.
• We have to decide how this looks in nimi, ilo, and lipu
• Even this extra work is far more doable if we change our workflow to divide words among smaller groups like we've started on with pan and esun

[lipamanka]

It's been a bit since you've written this and I'm not going to write a full response, but I would like to explain why I disagree with your central claim here.

from my pov, including broad definitions (let's say, for argument's sake, 30% usage) is about the same as grabbing a random word from linku's Uncommon category and bringing it up to Core.

that category includes apeja, epiku, jasima, kiki, kokosila, linluwi, majuna, and powe

I don't agree with this. using a word and not using a word is one thing. the broadness of scope of toki pona words is for many people very broad. Just because it's common to only use a word in some senses doesn't mean the less common senses should be in the same category as less common words. When I think of these less common usages of pu words, for example, these are all things I see people I know use all the time, because that's just what happens when you speak the language. You will end up using these words in lots of situations. In essence, I think these two statements are true.

• less common words are mostly used by beginners who don't actively use them to speak toki pona frequently
• less common usages of pu words are mostly used by speakers who use toki pona actively to speak with others

For these reasons I think it makes the most sense to include less common usages. I think it's really hard to measure actual usage. How do we know how often people filling out our forms use these senses of the words? We can't really measure that right now, but that's what I've seen a lot. I'd like to see if people agree with me on this.

[gregdan3]

less common usages of pu words are mostly used by speakers who use toki pona actively to speak with others

I don't think this claim works for me. To my experience, this is correct, but it also applies to a ton of less frequently used words- newbies make piles of words, use them for a short period, and tend to abandon them. Experienced speakers create words and continue using them, giving them more use, exploration, exercise, and sticking power.

In other words, "less common words [created by experienced speakers] are mostly used by speaker who use toki pona actively to speak with others." The same point, filtering for the kind of words we want to compare against, still applies to words and not just meanings of words.

I see your point that the usage of more experienced speakers should be considered more heavily, but I don't think that should be enough that experienced speakers' nasin directly informs what is "core" versus what is not. For better or worse, the average toki pona speaker IS less experienced, and we should respect that in the creation of a set of definitions to reflect the language- both for the purpose of teaching, and the purpose of truthfulness.

[lipamanka]

Experienced speakers create words and continue using them, giving them more use, exploration, exercise, and sticking power.

I don't think it's very descriptive to compare one person using a nimisin they made to differences in usage across extremely widespread words. I would like examples of this too. I don't think this is actually that common in practice, though it definitely happens.

I see your point that the usage of more experienced speakers should be considered more heavily, but I don't think that should be enough that experienced speakers' nasin directly informs what is "core" versus what is not.

my point here is that more proficiency scales directly with usage. more proficient speakers are more proficient because they use toki pona more, and people who are proficient tend to end up using it more. if we are describing instances of usage, most of the usage that does exist is between proficient speakers.

So if we want to be descriptive of how toki pona is used, and not descriptive of the self reporting without weighing for how much people use toki pona, we should try to account for that bias. We are approximating usage because we don't have like. a ton of data right now. 898 is a huge sample size but that doesn't mean the data we ended up collecting can accurately provide unbiased and unskewed results. If you have evidence that this bias is inaccurate and doesn't exist, please let me know. Otherwise I don't think it's a good idea to ignore it.

to sum up: more proficient speakers use toki pona more basically and also more proficient speakers are more likely to use "uncommon" senses of more common words.

[lipamanka]

I would like to say that your base idea here is cool, if we can support having "less common usage of pilin: to be in contact with" or something like that in the thing. but if we can't I would prefer to leave in less common usages, not leave them out. Though we can discuss exactly how we want to do that on a case-by-case basis, including mentioning it in the comments.

maybe I'll take a stats class by our next survey in a year and prepare for this kind of data collection lmao 💀