Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JSON serialization error in finetune.py #2

Open
KasperFyhn opened this issue Oct 6, 2023 · 0 comments
Open

JSON serialization error in finetune.py #2

KasperFyhn opened this issue Oct 6, 2023 · 0 comments
Assignees
Labels
bug Something isn't working

Comments

@KasperFyhn
Copy link

The below error occurs when the y argument for BertFinetuneClassifier.fit() contains a datatype that is not JSON serializable, e.g., if a labeled dataset contains 0's and 1's as labels and they are converted to int64 in a Numpy array.

File "~/llm-tweet-classification/utils/finetune.py", line 44, in fit
    self.model = AutoModelForSequenceClassification.from_pretrained(
  File "~/llm-tweet-classification/venv/lib/python3.10/site-packages/transformers/models/auto/auto_factory.py", line 525, in from_pretrained
    config, kwargs = AutoConfig.from_pretrained(
  File "~/llm-tweet-classification/venv/lib/python3.10/site-packages/transformers/models/auto/configuration_auto.py", line 1051, in from_pretrained
    return config_class.from_dict(config_dict, **unused_kwargs)
  File "~/llm-tweet-classification/venv/lib/python3.10/site-packages/transformers/configuration_utils.py", line 775, in from_dict
    logger.info(f"Model config {config}")
  File "~/llm-tweet-classification/venv/lib/python3.10/site-packages/transformers/configuration_utils.py", line 807, in __repr__
    return f"{self.__class__.__name__} {self.to_json_string()}"
  File "~/llm-tweet-classification/venv/lib/python3.10/site-packages/transformers/configuration_utils.py", line 918, in to_json_string
    return json.dumps(config_dict, indent=2, sort_keys=True) + "\n"
  File "/usr/lib/python3.10/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.10/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/usr/lib/python3.10/json/encoder.py", line 431, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.10/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.10/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/usr/lib/python3.10/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type int64 is not JSON serializable
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants