variables in dataset.py

[qfzhu]

Thanks for sharing the code.
May I ask what the variables "base_conv", "bias_conv", "base_nonc", and "bias_nonc" stand for?

[golsun]

hi @qfzhu ,
conv = "conversation"
nonc = "non-conversation"
base = background, non-stylized data
bias = corpus of the target style

[qfzhu]

Thanks for the rely.
I think there are only two types of data: parallel non-stylized conversational data (base_conv) and monolingual stylized data (bias_nonc).
What the "base_nonc" and "bias_conv" refer to,
how can I build them from the two types of data,
and are they necessary for training the model?

[golsun]

I think there are only two types of data: parallel non-stylized conversational data (base_conv) and monolingual stylized data (bias_nonc).

yes this is true for the implementation in the paper

What the "base_nonc" and "bias_conv" refer to,
how can I build them from the two types of data,
and are they necessary for training the model?

these two are optional (and we didn't use them in the paper).
bias_conv requires extraction of conversation from stylized corpus. this may be done for some corpus, e.g. Holmes novels (e.g. by regex), but not available for others, e.g. arXiv.
for base_nonc, you can simply use all the utterance turns from base_conv

[golsun]

close as questions are answered.