Baichuan2-13B does not have max_position_embeddings in config

[sywangyi]

crash like
2025-01-08T07:32:55.544276Z INFO shard-manager: text_generation_launcher: Starting shard rank=0
2025-01-08T07:32:59.391986Z INFO text_generation_launcher: Using prefix caching = True
2025-01-08T07:32:59.392009Z INFO text_generation_launcher: Using Attention = flashdecoding-ipex
2025-01-08T07:32:59.481495Z WARN text_generation_launcher: Could not import Mamba: No module named 'mamba_ssm'
2025-01-08T07:33:05.190189Z INFO text_generation_launcher: Using prefill chunking = True
2025-01-08T07:33:05.444738Z INFO text_generation_launcher: Server started at unix:///tmp/text-generation-server-0
2025-01-08T07:33:05.460390Z INFO shard-manager: text_generation_launcher: Shard ready in 9.908748294s rank=0
2025-01-08T07:33:05.552080Z INFO text_generation_launcher: Starting Webserver
2025-01-08T07:33:05.580980Z INFO text_generation_router_v3: backends/v3/src/lib.rs:125: Warming up model
2025-01-08T07:33:07.074059Z INFO text_generation_launcher: KV-cache blocks: 507, size: 64
2025-01-08T07:33:07.076651Z ERROR text_generation_launcher: Method Warmup encountered an error.
Traceback (most recent call last):
File "/opt/conda/bin/text-generation-server", line 8, in
sys.exit(app())
File "/opt/conda/lib/python3.11/site-packages/typer/main.py", line 321, in call
return get_command(self)(*args, **kwargs)
File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
File "/opt/conda/lib/python3.11/site-packages/typer/core.py", line 728, in main
return _main(
File "/opt/conda/lib/python3.11/site-packages/typer/core.py", line 197, in _main
rv = self.invoke(ctx)
File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/opt/conda/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/opt/conda/lib/python3.11/site-packages/typer/main.py", line 703, in wrapper
return callback(**use_params)
File "/opt/conda/lib/python3.11/site-packages/text_generation_server/cli.py", line 117, in serve
server.serve(
File "/opt/conda/lib/python3.11/site-packages/text_generation_server/server.py", line 315, in serve
asyncio.run(
File "/opt/conda/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
File "/opt/conda/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
File "/opt/conda/lib/python3.11/asyncio/base_events.py", line 641, in run_until_complete
self.run_forever()
File "/opt/conda/lib/python3.11/asyncio/base_events.py", line 608, in run_forever
self._run_once()
File "/opt/conda/lib/python3.11/asyncio/base_events.py", line 1936, in _run_once
handle._run()
File "/opt/conda/lib/python3.11/asyncio/events.py", line 84, in _run
self._context.run(self._callback, *self._args)
File "/opt/conda/lib/python3.11/site-packages/grpc_interceptor/server.py", line 165, in invoke_intercept_method
return await self.intercept(

File "/opt/conda/lib/python3.11/site-packages/text_generation_server/interceptor.py", line 24, in intercept
return await response
File "/opt/conda/lib/python3.11/site-packages/opentelemetry/instrumentation/grpc/_aio_server.py", line 120, in _unary_interceptor
raise error
File "/opt/conda/lib/python3.11/site-packages/opentelemetry/instrumentation/grpc/_aio_server.py", line 111, in _unary_interceptor
return await behavior(request_or_iterator, context)
File "/opt/conda/lib/python3.11/site-packages/text_generation_server/server.py", line 144, in Warmup
self.model.warmup(batch, max_input_tokens, max_total_tokens)
File "/opt/conda/lib/python3.11/site-packages/text_generation_server/models/flash_causal_lm.py", line 1598, in warmup
max_position_embeddings = self.config.max_position_embeddings
File "/opt/conda/lib/python3.11/site-packages/transformers/configuration_utils.py", line 205, in getattribute
return super().getattribute(key)
AttributeError: 'BaichuanConfig' object has no attribute 'max_position_embeddings'
2025-01-08T07:33:07.077290Z ERROR warmup{max_input_length=None max_prefill_tokens=4096 max_total_tokens=None max_batch_size=None}:warmup: text_generation_router_v3::client: backends/v3/src/client/mod.rs:45: Server error: 'BaichuanConfig' object has no attribute 'max_position_embeddings'
Error: Backend(Warmup(Generation("'BaichuanConfig' object has no attribute 'max_position_embeddings'")))
2025-01-08T07:33:07.162021Z ERROR text_generation_launcher: Webserver Crashed
2025-01-08T07:33:07.162035Z INFO text_generation_launcher: Shutting down shards
2025-01-08T07:33:07.261938Z INFO shard-manager: text_generation_launcher: Terminating shard rank=0
2025-01-08T07:33:07.261959Z INFO shard-manager: text_generation_launcher: Waiting for shard to gracefully shutdown rank=0
Error: WebserverFailed
2025-01-08T07:33:08.963278Z INFO shard-manager: text_generation_launcher: shard terminated rank=0

@OlivierDehaene OR @Narsil

[danieldk]

Good catch. Added a suggestion for simplifying.