While this works on VLLM 0.7.x, the latest one (0.8.2), which supports mistral-small and gemma, does not seem have the "engine" attribute:
2025-03-24 22:04:27 | ERROR | stderr | Traceback (most recent call last):
2025-03-24 22:04:27 | ERROR | stderr | File "/p/haicluster/llama/FastChat/fastchat/serve/vllm_worker.py", line 291, in <module>
2025-03-24 22:04:27 | ERROR | stderr | worker = VLLMWorker(
2025-03-24 22:04:27 | ERROR | stderr | ^^^^^^^^^^^
2025-03-24 22:04:27 | ERROR | stderr | File "/p/haicluster/llama/FastChat/fastchat/serve/vllm_worker.py", line 57, in __init__
2025-03-24 22:04:27 | ERROR | stderr | self.tokenizer = llm_engine.engine.tokenizer
2025-03-24 22:04:27 | ERROR | stderr | ^^^^^^^^^^^^^^^^^
2025-03-24 22:04:27 | ERROR | stderr | AttributeError: 'AsyncLLM' object has no attribute 'engine'
While this works on VLLM 0.7.x, the latest one (0.8.2), which supports mistral-small and gemma, does not seem have the "engine" attribute: