TypeError: 'NoneType' object is not subscriptable in modeling_utils.py

[MartinPicc]

System Info

Using free tier Google Colab, it gives the following output of transformers-cli env:

2023-03-07 12:26:45.314129: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia
2023-03-07 12:26:45.314255: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /usr/lib64-nvidia
2023-03-07 12:26:45.314280: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
WARNING:tensorflow:From /usr/local/lib/python3.8/dist-packages/transformers/commands/env.py:52: is_gpu_available (from tensorflow.python.framework.test_util) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.config.list_physical_devices('GPU') instead.
2023-03-07 12:26:49.528826: W tensorflow/core/common_runtime/gpu/gpu_bfc_allocator.cc:42] Overriding orig_value setting because the TF_FORCE_GPU_ALLOW_GROWTH environment variable is set. Original config value was 0.

Copy-and-paste the text below in your GitHub issue and FILL OUT the two last points.

• transformers version: 4.26.1
• Platform: Linux-5.10.147+-x86_64-with-glibc2.29
• Python version: 3.8.10
• Huggingface_hub version: 0.12.1
• PyTorch version (GPU?): 1.13.1+cu116 (True)
• Tensorflow version (GPU?): 2.11.0 (True)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: no
• Using distributed or parallel set-up in script?: no

Who can help?

@sgugger

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
• My own task or dataset (give details below)

Reproduction

Using the library Detoxify raises an error in transformers code if using transformers version >= 4.25.1, but works well with version 4.24 and below.

The error is: TypeError: 'NoneType' object is not subscriptable in file modeling_utils at line 2718. This line (and its block of code) has been added with PR#20321 merged in version 4.25.1

transformers/src/transformers/modeling_utils.py

Lines 2718 to 2736 in 820c46a

	folder = os.path.sep.join(resolved_archive_file[0].split(os.path.sep)[:-1])
	if device_map is not None and is_safetensors:
	param_device_map = expand_device_map(device_map, original_loaded_keys)
	str_dtype = str(dtype).replace("torch.", "") if dtype is not None else "float32"
	if sharded_metadata is None:
	archive_file = (
	resolved_archive_file[0]
	if isinstance(resolved_archive_file, (list, tuple))
	else resolved_archive_file
	)
	weight_map = {p: archive_file for p in original_loaded_keys}
	else:
	weight_map = {p: os.path.join(folder, f) for p, f in sharded_metadata["weight_map"].items()}
	offload_index = {
	p: {"safetensors_file": f, "weight_name": p, "dtype": str_dtype}
	for p, f in weight_map.items()
	if param_device_map[p] == "disk"
	}

Looking at the code, it seems to me that the variable resolved_archive_file can take the value None, hence raising this error.

The full error stacktrace is:

---------------------------------------------------------------------------

TypeError                                 Traceback (most recent call last)

<ipython-input-7-72e665f021e6> in <module>
----> 1 toxicity_model = Detoxify('multilingual', device='cuda')
      2 # results = toxicity_model.predict()

4 frames

/usr/local/lib/python3.8/dist-packages/detoxify/detoxify.py in __init__(self, model_type, checkpoint, device, huggingface_config_path)
    101     def __init__(self, model_type="original", checkpoint=PRETRAINED_MODEL, device="cpu", huggingface_config_path=None):
    102         super().__init__()
--> 103         self.model, self.tokenizer, self.class_names = load_checkpoint(
    104             model_type=model_type,
    105             checkpoint=checkpoint,

/usr/local/lib/python3.8/dist-packages/detoxify/detoxify.py in load_checkpoint(model_type, checkpoint, device, huggingface_config_path)
     54     }
     55     class_names = [change_names.get(cl, cl) for cl in class_names]
---> 56     model, tokenizer = get_model_and_tokenizer(
     57         **loaded["config"]["arch"]["args"],
     58         state_dict=loaded["state_dict"],

/usr/local/lib/python3.8/dist-packages/detoxify/detoxify.py in get_model_and_tokenizer(model_type, model_name, tokenizer_name, num_classes, state_dict, huggingface_config_path)
     18 ):
     19     model_class = getattr(transformers, model_name)
---> 20     model = model_class.from_pretrained(
     21         pretrained_model_name_or_path=None,
     22         config=huggingface_config_path or model_type,

/usr/local/lib/python3.8/dist-packages/transformers/modeling_utils.py in from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
   2476                 offload_index,
   2477                 error_msgs,
-> 2478             ) = cls._load_pretrained_model(
   2479                 model,
   2480                 state_dict,

/usr/local/lib/python3.8/dist-packages/transformers/modeling_utils.py in _load_pretrained_model(cls, model, state_dict, loaded_keys, resolved_archive_file, pretrained_model_name_or_path, ignore_mismatched_sizes, sharded_metadata, _fast_init, low_cpu_mem_usage, device_map, offload_folder, offload_state_dict, dtype, load_in_8bit, keep_in_fp32_modules)
   2716             return mismatched_keys
   2717 
-> 2718         folder = os.path.sep.join(resolved_archive_file[0].split(os.path.sep)[:-1])
   2719         if device_map is not None and is_safetensors:
   2720             param_device_map = expand_device_map(device_map, original_loaded_keys)

TypeError: 'NoneType' object is not subscriptable

PS: link to the related issue in the library Detoxify

Expected behavior

Put a condition on resolved_archive_file to handle the case when its value is None.
However, if its value SHOULDN'T be None, then add a validity check earlier in the code, with more explicit details.

Let me know if I can help on this.

[sgugger]

I think this has been fixed by #21542. Could you try on the main branch of Transformers and see if you still have the bug?

[MartinPicc]

Oh yes perfect! I'll wait for the next release to update then.
thank you

[sgugger]

Next release should be this week or beginning of next, as an FYI :-)

[davicompu]

I am still having the same problem when running Detoxify. The code I implemented is below:

from detoxify import Detoxify
import pandas as pd
# text variable and value are hidden
results = Detoxify('original').predict(text)

Please advise on how I can make it work.

[ArthurZucker]

This is unrelated to transformers 😉

[davicompu]

Thank you, @ArthurZucker, for your quick response. Could you point me in the right direction, please? What do I need to do to make Detoxify work?

[ArthurZucker]

I would submit the issue to detoxify library on github!