/usr/lib64/python3.11/inspect.py:389: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(object, types.FunctionType)
Detected capabilities: [-cpu -gaudi -gaudi2 +gaudi3 +index_reduce]
INFO 01-31 03:20:49 api_server.py:592] vLLM API server version 0.6.4.post2
INFO 01-31 03:20:49 api_server.py:593] args: Namespace(subparser='serve', model_tag='instructlab/granite-7b-lab', config='', host=None, port=8000, uvicorn_log_level='info', allow_credentials=False, allowed_origins=['*'], allowed_methods=['*'], allowed_headers=['*'], api_key=None, lora_modules=None, prompt_adapters=None, chat_template=None, chat_template_content_format='auto', response_role='assistant', ssl_keyfile=None, ssl_certfile=None, ssl_ca_certs=None, ssl_cert_reqs=0, root_path=None, middleware=[], return_tokens_as_token_ids=False, disable_frontend_multiprocessing=False, enable_auto_tool_choice=False, tool_call_parser=None, tool_parser_plugin='', model='instructlab/granite-7b-lab', task='auto', tokenizer=None, skip_tokenizer_init=False, revision=None, code_revision=None, tokenizer_revision=None, tokenizer_mode='auto', trust_remote_code=False, allowed_local_media_path=None, download_dir=None, load_format='auto', weights_load_device=None, config_format=<ConfigFormat.AUTO: 'auto'>, dtype='bfloat16', kv_cache_dtype='auto', quantization_param_path=None, max_model_len=None, guided_decoding_backend='outlines', distributed_executor_backend=None, worker_use_ray=False, pipeline_parallel_size=1, tensor_parallel_size=2, max_parallel_loading_workers=None, ray_workers_use_nsight=False, block_size=128, enable_prefix_caching=False, disable_sliding_window=False, use_v2_block_manager=False, use_padding_aware_scheduling=False, num_lookahead_slots=0, seed=0, swap_space=4, cpu_offload_gb=0, gpu_memory_utilization=0.9, num_gpu_blocks_override=None, max_num_batched_tokens=None, max_num_seqs=256, max_num_prefill_seqs=None, max_logprobs=20, disable_log_stats=False, quantization=None, rope_scaling=None, rope_theta=None, hf_overrides=None, enforce_eager=False, max_seq_len_to_capture=8192, disable_custom_all_reduce=False, tokenizer_pool_size=0, tokenizer_pool_type='ray', tokenizer_pool_extra_config=None, limit_mm_per_prompt=None, mm_processor_kwargs=None, enable_lora=False, enable_lora_bias=False, max_loras=1, max_lora_rank=16, lora_extra_vocab_size=256, lora_dtype='auto', long_lora_scaling_factors=None, max_cpu_loras=None, fully_sharded_loras=False, enable_prompt_adapter=False, max_prompt_adapters=1, max_prompt_adapter_token=0, device='auto', num_scheduler_steps=1, multi_step_stream_outputs=True, scheduler_delay_factor=0.0, enable_chunked_prefill=None, speculative_model=None, speculative_model_quantization=None, num_speculative_tokens=None, speculative_disable_mqa_scorer=False, speculative_draft_tensor_parallel_size=None, speculative_max_model_len=None, speculative_disable_by_batch_size=None, ngram_prompt_lookup_max=None, ngram_prompt_lookup_min=None, spec_decoding_acceptance_method='rejection_sampler', typical_acceptance_sampler_posterior_threshold=None, typical_acceptance_sampler_posterior_alpha=None, disable_logprobs_during_spec_decoding=None, model_loader_extra_config=None, ignore_patterns=[], preemption_mode=None, served_model_name=None, qlora_adapter_name_or_path=None, otlp_traces_endpoint=None, collect_detailed_traces=None, disable_async_output_proc=False, scheduling_policy='fcfs', override_neuron_config=None, override_pooler_config=None, disable_log_requests=False, max_log_len=None, disable_fastapi_docs=False, enable_prompt_tokens_details=False, dispatch_function=<function serve at 0x7f947d9200e0>)
INFO 01-31 03:20:49 __init__.py:31] No plugins found.
INFO 01-31 03:20:49 api_server.py:176] Multiprocessing frontend to use ipc:///tmp/abc65108-fbe2-4099-b87b-69149b58d335 for IPC Path.
INFO 01-31 03:20:49 api_server.py:195] Started engine process with PID 19558
INFO 01-31 03:20:50 config.py:1874] Downcasting torch.float32 to torch.bfloat16.
/usr/lib64/python3.11/inspect.py:389: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(object, types.FunctionType)
Detected capabilities: [-cpu -gaudi -gaudi2 +gaudi3 +index_reduce]
INFO 01-31 03:20:55 __init__.py:31] No plugins found.
INFO 01-31 03:20:56 config.py:1874] Downcasting torch.float32 to torch.bfloat16.
INFO 01-31 03:20:57 config.py:350] This model supports multiple tasks: {'embedding', 'generate'}. Defaulting to 'generate'.
INFO 01-31 03:20:57 config.py:1017] Defaulting to use mp for distributed inference
WARNING 01-31 03:20:57 arg_utils.py:1092] [DEPRECATED] Block manager v1 has been removed, and setting --use-v2-block-manager to True or False has no effect on vLLM behavior. Please remove --use-v2-block-manager in your engine argument. If your use case is not supported by SelfAttnBlockSpaceManager (i.e. block manager v2), please file an issue with detailed information.
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama_fast.LlamaTokenizerFast'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message.
INFO 01-31 03:21:04 config.py:350] This model supports multiple tasks: {'embedding', 'generate'}. Defaulting to 'generate'.
INFO 01-31 03:21:04 config.py:1017] Defaulting to use mp for distributed inference
WARNING 01-31 03:21:04 arg_utils.py:1092] [DEPRECATED] Block manager v1 has been removed, and setting --use-v2-block-manager to True or False has no effect on vLLM behavior. Please remove --use-v2-block-manager in your engine argument. If your use case is not supported by SelfAttnBlockSpaceManager (i.e. block manager v2), please file an issue with detailed information.
INFO 01-31 03:21:04 llm_engine.py:250] Initializing an LLM engine (v0.6.4.post2) with config: model='instructlab/granite-7b-lab', speculative_config=None, tokenizer='instructlab/granite-7b-lab', skip_tokenizer_init=False, tokenizer_mode=auto, revision=None, override_neuron_config=None, tokenizer_revision=None, trust_remote_code=False, dtype=torch.bfloat16, max_seq_len=4096, download_dir=None, load_format=LoadFormat.AUTO, tensor_parallel_size=2, pipeline_parallel_size=1, disable_custom_all_reduce=False, quantization=None, weights_load_device=hpu, enforce_eager=False, kv_cache_dtype=auto, quantization_param_path=None, device_config=hpu, decoding_config=DecodingConfig(guided_decoding_backend='outlines'), observability_config=ObservabilityConfig(otlp_traces_endpoint=None, collect_model_forward_time=False, collect_model_execute_time=False), seed=0, served_model_name=instructlab/granite-7b-lab, num_scheduler_steps=1, chunked_prefill_enabled=False multi_step_stream_outputs=True, enable_prefix_caching=False, use_async_output_proc=True, use_cached_outputs=True, mm_processor_kwargs=None, pooler_config=None)
You are using the default legacy behaviour of the <class 'transformers.models.llama.tokenization_llama_fast.LlamaTokenizerFast'>. This is expected, and simply means that the `legacy` (previous) behavior will be used so nothing changes for you. If you want to use the new behaviour, set `legacy=False`. This should only be set if you understand what it means, and thoroughly read the reason why this was added as explained in https://github.com/huggingface/transformers/pull/24565 - if you loaded a llama tokenizer from a GGUF file you can ignore this message.
WARNING 01-31 03:21:04 multiproc_gpu_executor.py:56] Reducing Torch parallelism from 144 threads to 1 to avoid unnecessary CPU contention. Set OMP_NUM_THREADS in the external environment to tune this value as needed.
INFO 01-31 03:21:04 custom_cache_manager.py:17] Setting Triton cache manager to: vllm.triton_utils.custom_cache_manager:CustomCacheManager
INFO 01-31 03:21:04 __init__.py:31] No plugins found.
WARNING 01-31 03:21:04 utils.py:754] Pin memory is not supported on HPU.
INFO 01-31 03:21:04 selector.py:174] Using HPUAttention backend.
============================= HABANA PT BRIDGE CONFIGURATION =========================== 
 PT_HPU_LAZY_MODE = 1
 PT_RECIPE_CACHE_PATH = 
 PT_CACHE_FOLDER_DELETE = 0
 PT_HPU_RECIPE_CACHE_CONFIG = 
 PT_HPU_MAX_COMPOUND_OP_SIZE = 9223372036854775807
 PT_HPU_LAZY_ACC_PAR_MODE = 1
 PT_HPU_ENABLE_REFINE_DYNAMIC_SHAPES = 0
 PT_HPU_EAGER_PIPELINE_ENABLE = 1
 PT_HPU_EAGER_COLLECTIVE_PIPELINE_ENABLE = 1
---------------------------: System Configuration :---------------------------
Num CPU Cores : 288
CPU RAM       : 1979158020 KB
------------------------------------------------------------------------------
/usr/lib64/python3.11/inspect.py:389: FutureWarning: `torch.distributed.reduce_op` is deprecated, please use `torch.distributed.ReduceOp` instead
  return isinstance(object, types.FunctionType)
Detected capabilities: [-cpu -gaudi -gaudi2 +gaudi3 +index_reduce]
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:10 __init__.py:31] No plugins found.
(VllmWorkerProcess pid=19699) WARNING 01-31 03:21:11 utils.py:754] Pin memory is not supported on HPU.
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:11 selector.py:174] Using HPUAttention backend.
(VllmWorkerProcess pid=19699) VLLM_PROMPT_BS_BUCKET_MIN=1 (default:1)
(VllmWorkerProcess pid=19699) VLLM_PROMPT_BS_BUCKET_STEP=32 (default:32)
(VllmWorkerProcess pid=19699) VLLM_PROMPT_BS_BUCKET_MAX=256 (default:256)
(VllmWorkerProcess pid=19699) VLLM_DECODE_BS_BUCKET_MIN=1 (default:1)
(VllmWorkerProcess pid=19699) VLLM_DECODE_BS_BUCKET_STEP=32 (default:32)
(VllmWorkerProcess pid=19699) VLLM_DECODE_BS_BUCKET_MAX=256 (default:256)
(VllmWorkerProcess pid=19699) VLLM_PROMPT_SEQ_BUCKET_MIN=128 (default:128)
(VllmWorkerProcess pid=19699) VLLM_PROMPT_SEQ_BUCKET_STEP=128 (default:128)
(VllmWorkerProcess pid=19699) VLLM_PROMPT_SEQ_BUCKET_MAX=1024 (default:1024)
(VllmWorkerProcess pid=19699) VLLM_DECODE_BLOCK_BUCKET_MIN=128 (default:128)
(VllmWorkerProcess pid=19699) VLLM_DECODE_BLOCK_BUCKET_STEP=128 (default:128)
(VllmWorkerProcess pid=19699) VLLM_DECODE_BLOCK_BUCKET_MAX=4096 (default:4096)
(VllmWorkerProcess pid=19699) Prompt bucket config (min, step, max_warmup) bs:[1, 32, 256], seq:[128, 128, 1024]
(VllmWorkerProcess pid=19699) Decode bucket config (min, step, max_warmup) bs:[1, 32, 256], block:[128, 128, 4096]
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:11 multiproc_worker_utils.py:215] Worker ready; awaiting tasks
============================= HABANA PT BRIDGE CONFIGURATION =========================== 
 PT_HPU_LAZY_MODE = 1
 PT_RECIPE_CACHE_PATH = 
 PT_CACHE_FOLDER_DELETE = 0
 PT_HPU_RECIPE_CACHE_CONFIG = 
 PT_HPU_MAX_COMPOUND_OP_SIZE = 9223372036854775807
 PT_HPU_LAZY_ACC_PAR_MODE = 1
 PT_HPU_ENABLE_REFINE_DYNAMIC_SHAPES = 0
 PT_HPU_EAGER_PIPELINE_ENABLE = 1
 PT_HPU_EAGER_COLLECTIVE_PIPELINE_ENABLE = 1
---------------------------: System Configuration :---------------------------
Num CPU Cores : 288
CPU RAM       : 1979158020 KB
------------------------------------------------------------------------------
VLLM_PROMPT_BS_BUCKET_MIN=1 (default:1)
VLLM_PROMPT_BS_BUCKET_STEP=32 (default:32)
VLLM_PROMPT_BS_BUCKET_MAX=256 (default:256)
VLLM_DECODE_BS_BUCKET_MIN=1 (default:1)
VLLM_DECODE_BS_BUCKET_STEP=32 (default:32)
VLLM_DECODE_BS_BUCKET_MAX=256 (default:256)
VLLM_PROMPT_SEQ_BUCKET_MIN=128 (default:128)
VLLM_PROMPT_SEQ_BUCKET_STEP=128 (default:128)
VLLM_PROMPT_SEQ_BUCKET_MAX=1024 (default:1024)
VLLM_DECODE_BLOCK_BUCKET_MIN=128 (default:128)
VLLM_DECODE_BLOCK_BUCKET_STEP=128 (default:128)
VLLM_DECODE_BLOCK_BUCKET_MAX=4096 (default:4096)
Prompt bucket config (min, step, max_warmup) bs:[1, 32, 256], seq:[128, 128, 1024]
Decode bucket config (min, step, max_warmup) bs:[1, 32, 256], block:[128, 128, 4096]
INFO 01-31 03:21:12 shm_broadcast.py:236] vLLM message queue communication handle: Handle(connect_ip='127.0.0.1', local_reader_ranks=[1], buffer=<vllm.distributed.device_communicators.shm_broadcast.ShmRingBuffer object at 0x7fd7fa130490>, local_subscribe_port=37221, remote_subscribe_port=None)
INFO 01-31 03:21:12 loader.py:340] Loading weights on hpu...
INFO 01-31 03:21:13 weight_utils.py:243] Using model weights format ['*.safetensors']
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:13 loader.py:340] Loading weights on hpu...

Loading safetensors checkpoint shards:   0% Completed | 0/3 [00:00<?, ?it/s]
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:13 weight_utils.py:243] Using model weights format ['*.safetensors']

Loading safetensors checkpoint shards:  33% Completed | 1/3 [00:01<00:03,  1.51s/it]

Loading safetensors checkpoint shards:  67% Completed | 2/3 [00:02<00:01,  1.06s/it]

Loading safetensors checkpoint shards: 100% Completed | 3/3 [00:03<00:00,  1.05it/s]

Loading safetensors checkpoint shards: 100% Completed | 3/3 [00:03<00:00,  1.03s/it]

INFO 01-31 03:21:16 hpu_model_runner.py:611] Pre-loading model weights on hpu:0 took 6.277 GiB of device memory (6.284 GiB/126.6 GiB used) and 270.9 MiB of host memory (83.68 GiB/1.843 TiB used)
INFO 01-31 03:21:16 hpu_model_runner.py:683] Wrapping in HPU Graph took 0 B of device memory (6.284 GiB/126.6 GiB used) and 0 B of host memory (83.67 GiB/1.843 TiB used)
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:16 hpu_model_runner.py:611] Pre-loading model weights on hpu:0 took 6.277 GiB of device memory (6.284 GiB/126.6 GiB used) and 253.3 MiB of host memory (83.67 GiB/1.843 TiB used)
INFO 01-31 03:21:17 hpu_model_runner.py:687] Loading model weights took in total 6.277 GiB of device memory (6.284 GiB/126.6 GiB used) and 256.3 MiB of host memory (83.67 GiB/1.843 TiB used)
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:17 hpu_model_runner.py:683] Wrapping in HPU Graph took 0 B of device memory (6.284 GiB/126.6 GiB used) and 0 B of host memory (83.67 GiB/1.843 TiB used)
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:17 hpu_model_runner.py:687] Loading model weights took in total 6.277 GiB of device memory (6.284 GiB/126.6 GiB used) and 253.3 MiB of host memory (83.67 GiB/1.843 TiB used)
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:19 hpu_worker.py:177] Model profiling run took 108 MiB of device memory (6.39 GiB/126.6 GiB used) and 94.31 MiB of host memory (83.76 GiB/1.843 TiB used)
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:19 hpu_worker.py:201] Free device memory: 120.2 GiB, 108.2 GiB usable (gpu_memory_utilization=0.9), 10.82 GiB reserved for HPUGraphs (VLLM_GRAPH_RESERVED_MEM=0.1), 97.39 GiB reserved for KV cache
INFO 01-31 03:21:19 hpu_worker.py:177] Model profiling run took 108 MiB of device memory (6.39 GiB/126.6 GiB used) and 99.72 MiB of host memory (83.76 GiB/1.843 TiB used)
INFO 01-31 03:21:19 hpu_worker.py:201] Free device memory: 120.2 GiB, 108.2 GiB usable (gpu_memory_utilization=0.9), 10.82 GiB reserved for HPUGraphs (VLLM_GRAPH_RESERVED_MEM=0.1), 97.39 GiB reserved for KV cache
INFO 01-31 03:21:19 distributed_gpu_executor.py:57] # GPU blocks: 3116, # CPU blocks: 128
INFO 01-31 03:21:19 distributed_gpu_executor.py:61] Maximum concurrency for 4096 tokens per request: 97.38x
INFO 01-31 03:21:22 hpu_worker.py:234] Initializing cache engine took 97.38 GiB of device memory (103.8 GiB/126.6 GiB used) and 8.016 GiB of host memory (91.78 GiB/1.843 TiB used)
Generated 31 prompt buckets [bs, seq]: [(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (8, 128), (8, 256), (8, 384), (8, 512), (16, 128), (16, 256), (32, 128)]
Omitted 73 prompt buckets due to exceeded token budget (max_num_batched_tokens=4096)
Omitted prompt buckets: [(8, 640), (8, 768), (8, 896), (8, 1024), (16, 384), (16, 512), (16, 640), (16, 768), (16, 896), (16, 1024), (32, 256), (32, 384), (32, 512), (32, 640), (32, 768), (32, 896), (32, 1024), (64, 128), (64, 256), (64, 384), (64, 512), (64, 640), (64, 768), (64, 896), (64, 1024), (96, 128), (96, 256), (96, 384), (96, 512), (96, 640), (96, 768), (96, 896), (96, 1024), (128, 128), (128, 256), (128, 384), (128, 512), (128, 640), (128, 768), (128, 896), (128, 1024), (160, 128), (160, 256), (160, 384), (160, 512), (160, 640), (160, 768), (160, 896), (160, 1024), (192, 128), (192, 256), (192, 384), (192, 512), (192, 640), (192, 768), (192, 896), (192, 1024), (224, 128), (224, 256), (224, 384), (224, 512), (224, 640), (224, 768), (224, 896), (224, 1024), (256, 128), (256, 256), (256, 384), (256, 512), (256, 640), (256, 768), (256, 896), (256, 1024)]
Generated 325 decode buckets [bs, total_blocks]: [(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (1, 1152), (1, 1280), (1, 1408), (1, 1536), (1, 1664), (1, 1792), (1, 1920), (1, 2048), (1, 2176), (1, 2304), (1, 2432), (1, 2560), (1, 2688), (1, 2816), (1, 2944), (1, 3072), (1, 3116), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (2, 1152), (2, 1280), (2, 1408), (2, 1536), (2, 1664), (2, 1792), (2, 1920), (2, 2048), (2, 2176), (2, 2304), (2, 2432), (2, 2560), (2, 2688), (2, 2816), (2, 2944), (2, 3072), (2, 3116), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (4, 1152), (4, 1280), (4, 1408), (4, 1536), (4, 1664), (4, 1792), (4, 1920), (4, 2048), (4, 2176), (4, 2304), (4, 2432), (4, 2560), (4, 2688), (4, 2816), (4, 2944), (4, 3072), (4, 3116), (8, 128), (8, 256), (8, 384), (8, 512), (8, 640), (8, 768), (8, 896), (8, 1024), (8, 1152), (8, 1280), (8, 1408), (8, 1536), (8, 1664), (8, 1792), (8, 1920), (8, 2048), (8, 2176), (8, 2304), (8, 2432), (8, 2560), (8, 2688), (8, 2816), (8, 2944), (8, 3072), (8, 3116), (16, 128), (16, 256), (16, 384), (16, 512), (16, 640), (16, 768), (16, 896), (16, 1024), (16, 1152), (16, 1280), (16, 1408), (16, 1536), (16, 1664), (16, 1792), (16, 1920), (16, 2048), (16, 2176), (16, 2304), (16, 2432), (16, 2560), (16, 2688), (16, 2816), (16, 2944), (16, 3072), (16, 3116), (32, 128), (32, 256), (32, 384), (32, 512), (32, 640), (32, 768), (32, 896), (32, 1024), (32, 1152), (32, 1280), (32, 1408), (32, 1536), (32, 1664), (32, 1792), (32, 1920), (32, 2048), (32, 2176), (32, 2304), (32, 2432), (32, 2560), (32, 2688), (32, 2816), (32, 2944), (32, 3072), (32, 3116), (64, 128), (64, 256), (64, 384), (64, 512), (64, 640), (64, 768), (64, 896), (64, 1024), (64, 1152), (64, 1280), (64, 1408), (64, 1536), (64, 1664), (64, 1792), (64, 1920), (64, 2048), (64, 2176), (64, 2304), (64, 2432), (64, 2560), (64, 2688), (64, 2816), (64, 2944), (64, 3072), (64, 3116), (96, 128), (96, 256), (96, 384), (96, 512), (96, 640), (96, 768), (96, 896), (96, 1024), (96, 1152), (96, 1280), (96, 1408), (96, 1536), (96, 1664), (96, 1792), (96, 1920), (96, 2048), (96, 2176), (96, 2304), (96, 2432), (96, 2560), (96, 2688), (96, 2816), (96, 2944), (96, 3072), (96, 3116), (128, 128), (128, 256), (128, 384), (128, 512), (128, 640), (128, 768), (128, 896), (128, 1024), (128, 1152), (128, 1280), (128, 1408), (128, 1536), (128, 1664), (128, 1792), (128, 1920), (128, 2048), (128, 2176), (128, 2304), (128, 2432), (128, 2560), (128, 2688), (128, 2816), (128, 2944), (128, 3072), (128, 3116), (160, 128), (160, 256), (160, 384), (160, 512), (160, 640), (160, 768), (160, 896), (160, 1024), (160, 1152), (160, 1280), (160, 1408), (160, 1536), (160, 1664), (160, 1792), (160, 1920), (160, 2048), (160, 2176), (160, 2304), (160, 2432), (160, 2560), (160, 2688), (160, 2816), (160, 2944), (160, 3072), (160, 3116), (192, 128), (192, 256), (192, 384), (192, 512), (192, 640), (192, 768), (192, 896), (192, 1024), (192, 1152), (192, 1280), (192, 1408), (192, 1536), (192, 1664), (192, 1792), (192, 1920), (192, 2048), (192, 2176), (192, 2304), (192, 2432), (192, 2560), (192, 2688), (192, 2816), (192, 2944), (192, 3072), (192, 3116), (224, 128), (224, 256), (224, 384), (224, 512), (224, 640), (224, 768), (224, 896), (224, 1024), (224, 1152), (224, 1280), (224, 1408), (224, 1536), (224, 1664), (224, 1792), (224, 1920), (224, 2048), (224, 2176), (224, 2304), (224, 2432), (224, 2560), (224, 2688), (224, 2816), (224, 2944), (224, 3072), (224, 3116), (256, 128), (256, 256), (256, 384), (256, 512), (256, 640), (256, 768), (256, 896), (256, 1024), (256, 1152), (256, 1280), (256, 1408), (256, 1536), (256, 1664), (256, 1792), (256, 1920), (256, 2048), (256, 2176), (256, 2304), (256, 2432), (256, 2560), (256, 2688), (256, 2816), (256, 2944), (256, 3072), (256, 3116)]
INFO 01-31 03:21:22 hpu_model_runner.py:1473] [Warmup][Prompt][1/31] batch_size:4 seq_len:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:22 hpu_worker.py:234] Initializing cache engine took 97.38 GiB of device memory (103.8 GiB/126.6 GiB used) and 7.476 GiB of host memory (91.78 GiB/1.843 TiB used)
(VllmWorkerProcess pid=19699) Generated 31 prompt buckets [bs, seq]: [(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (8, 128), (8, 256), (8, 384), (8, 512), (16, 128), (16, 256), (32, 128)]
(VllmWorkerProcess pid=19699) Omitted 73 prompt buckets due to exceeded token budget (max_num_batched_tokens=4096)
(VllmWorkerProcess pid=19699) Omitted prompt buckets: [(8, 640), (8, 768), (8, 896), (8, 1024), (16, 384), (16, 512), (16, 640), (16, 768), (16, 896), (16, 1024), (32, 256), (32, 384), (32, 512), (32, 640), (32, 768), (32, 896), (32, 1024), (64, 128), (64, 256), (64, 384), (64, 512), (64, 640), (64, 768), (64, 896), (64, 1024), (96, 128), (96, 256), (96, 384), (96, 512), (96, 640), (96, 768), (96, 896), (96, 1024), (128, 128), (128, 256), (128, 384), (128, 512), (128, 640), (128, 768), (128, 896), (128, 1024), (160, 128), (160, 256), (160, 384), (160, 512), (160, 640), (160, 768), (160, 896), (160, 1024), (192, 128), (192, 256), (192, 384), (192, 512), (192, 640), (192, 768), (192, 896), (192, 1024), (224, 128), (224, 256), (224, 384), (224, 512), (224, 640), (224, 768), (224, 896), (224, 1024), (256, 128), (256, 256), (256, 384), (256, 512), (256, 640), (256, 768), (256, 896), (256, 1024)]
(VllmWorkerProcess pid=19699) Generated 325 decode buckets [bs, total_blocks]: [(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (1, 1152), (1, 1280), (1, 1408), (1, 1536), (1, 1664), (1, 1792), (1, 1920), (1, 2048), (1, 2176), (1, 2304), (1, 2432), (1, 2560), (1, 2688), (1, 2816), (1, 2944), (1, 3072), (1, 3116), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (2, 1152), (2, 1280), (2, 1408), (2, 1536), (2, 1664), (2, 1792), (2, 1920), (2, 2048), (2, 2176), (2, 2304), (2, 2432), (2, 2560), (2, 2688), (2, 2816), (2, 2944), (2, 3072), (2, 3116), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (4, 1152), (4, 1280), (4, 1408), (4, 1536), (4, 1664), (4, 1792), (4, 1920), (4, 2048), (4, 2176), (4, 2304), (4, 2432), (4, 2560), (4, 2688), (4, 2816), (4, 2944), (4, 3072), (4, 3116), (8, 128), (8, 256), (8, 384), (8, 512), (8, 640), (8, 768), (8, 896), (8, 1024), (8, 1152), (8, 1280), (8, 1408), (8, 1536), (8, 1664), (8, 1792), (8, 1920), (8, 2048), (8, 2176), (8, 2304), (8, 2432), (8, 2560), (8, 2688), (8, 2816), (8, 2944), (8, 3072), (8, 3116), (16, 128), (16, 256), (16, 384), (16, 512), (16, 640), (16, 768), (16, 896), (16, 1024), (16, 1152), (16, 1280), (16, 1408), (16, 1536), (16, 1664), (16, 1792), (16, 1920), (16, 2048), (16, 2176), (16, 2304), (16, 2432), (16, 2560), (16, 2688), (16, 2816), (16, 2944), (16, 3072), (16, 3116), (32, 128), (32, 256), (32, 384), (32, 512), (32, 640), (32, 768), (32, 896), (32, 1024), (32, 1152), (32, 1280), (32, 1408), (32, 1536), (32, 1664), (32, 1792), (32, 1920), (32, 2048), (32, 2176), (32, 2304), (32, 2432), (32, 2560), (32, 2688), (32, 2816), (32, 2944), (32, 3072), (32, 3116), (64, 128), (64, 256), (64, 384), (64, 512), (64, 640), (64, 768), (64, 896), (64, 1024), (64, 1152), (64, 1280), (64, 1408), (64, 1536), (64, 1664), (64, 1792), (64, 1920), (64, 2048), (64, 2176), (64, 2304), (64, 2432), (64, 2560), (64, 2688), (64, 2816), (64, 2944), (64, 3072), (64, 3116), (96, 128), (96, 256), (96, 384), (96, 512), (96, 640), (96, 768), (96, 896), (96, 1024), (96, 1152), (96, 1280), (96, 1408), (96, 1536), (96, 1664), (96, 1792), (96, 1920), (96, 2048), (96, 2176), (96, 2304), (96, 2432), (96, 2560), (96, 2688), (96, 2816), (96, 2944), (96, 3072), (96, 3116), (128, 128), (128, 256), (128, 384), (128, 512), (128, 640), (128, 768), (128, 896), (128, 1024), (128, 1152), (128, 1280), (128, 1408), (128, 1536), (128, 1664), (128, 1792), (128, 1920), (128, 2048), (128, 2176), (128, 2304), (128, 2432), (128, 2560), (128, 2688), (128, 2816), (128, 2944), (128, 3072), (128, 3116), (160, 128), (160, 256), (160, 384), (160, 512), (160, 640), (160, 768), (160, 896), (160, 1024), (160, 1152), (160, 1280), (160, 1408), (160, 1536), (160, 1664), (160, 1792), (160, 1920), (160, 2048), (160, 2176), (160, 2304), (160, 2432), (160, 2560), (160, 2688), (160, 2816), (160, 2944), (160, 3072), (160, 3116), (192, 128), (192, 256), (192, 384), (192, 512), (192, 640), (192, 768), (192, 896), (192, 1024), (192, 1152), (192, 1280), (192, 1408), (192, 1536), (192, 1664), (192, 1792), (192, 1920), (192, 2048), (192, 2176), (192, 2304), (192, 2432), (192, 2560), (192, 2688), (192, 2816), (192, 2944), (192, 3072), (192, 3116), (224, 128), (224, 256), (224, 384), (224, 512), (224, 640), (224, 768), (224, 896), (224, 1024), (224, 1152), (224, 1280), (224, 1408), (224, 1536), (224, 1664), (224, 1792), (224, 1920), (224, 2048), (224, 2176), (224, 2304), (224, 2432), (224, 2560), (224, 2688), (224, 2816), (224, 2944), (224, 3072), (224, 3116), (256, 128), (256, 256), (256, 384), (256, 512), (256, 640), (256, 768), (256, 896), (256, 1024), (256, 1152), (256, 1280), (256, 1408), (256, 1536), (256, 1664), (256, 1792), (256, 1920), (256, 2048), (256, 2176), (256, 2304), (256, 2432), (256, 2560), (256, 2688), (256, 2816), (256, 2944), (256, 3072), (256, 3116)]
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:22 hpu_model_runner.py:1473] [Warmup][Prompt][1/31] batch_size:4 seq_len:1024 free_mem:22.86 GiB
INFO 01-31 03:21:23 hpu_model_runner.py:1473] [Warmup][Prompt][2/31] batch_size:8 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:23 hpu_model_runner.py:1473] [Warmup][Prompt][2/31] batch_size:8 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:24 hpu_model_runner.py:1473] [Warmup][Prompt][3/31] batch_size:16 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:21:24 hpu_model_runner.py:1473] [Warmup][Prompt][3/31] batch_size:16 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:25 hpu_model_runner.py:1473] [Warmup][Prompt][4/31] batch_size:32 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:21:26 hpu_model_runner.py:1473] [Warmup][Prompt][4/31] batch_size:32 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:27 hpu_model_runner.py:1473] [Warmup][Prompt][5/31] batch_size:4 seq_len:896 free_mem:22.86 GiB
INFO 01-31 03:21:27 hpu_model_runner.py:1473] [Warmup][Prompt][5/31] batch_size:4 seq_len:896 free_mem:22.86 GiB
INFO 01-31 03:21:28 hpu_model_runner.py:1473] [Warmup][Prompt][6/31] batch_size:4 seq_len:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:28 hpu_model_runner.py:1473] [Warmup][Prompt][6/31] batch_size:4 seq_len:768 free_mem:22.86 GiB
INFO 01-31 03:21:29 hpu_model_runner.py:1473] [Warmup][Prompt][7/31] batch_size:8 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:29 hpu_model_runner.py:1473] [Warmup][Prompt][7/31] batch_size:8 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:21:30 hpu_model_runner.py:1473] [Warmup][Prompt][8/31] batch_size:4 seq_len:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:30 hpu_model_runner.py:1473] [Warmup][Prompt][8/31] batch_size:4 seq_len:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:31 hpu_model_runner.py:1473] [Warmup][Prompt][9/31] batch_size:2 seq_len:1024 free_mem:22.86 GiB
INFO 01-31 03:21:32 hpu_model_runner.py:1473] [Warmup][Prompt][9/31] batch_size:2 seq_len:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:33 hpu_model_runner.py:1473] [Warmup][Prompt][10/31] batch_size:4 seq_len:512 free_mem:22.86 GiB
INFO 01-31 03:21:33 hpu_model_runner.py:1473] [Warmup][Prompt][10/31] batch_size:4 seq_len:512 free_mem:22.86 GiB
INFO 01-31 03:21:34 hpu_model_runner.py:1473] [Warmup][Prompt][11/31] batch_size:8 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:34 hpu_model_runner.py:1473] [Warmup][Prompt][11/31] batch_size:8 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:35 hpu_model_runner.py:1473] [Warmup][Prompt][12/31] batch_size:16 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:21:35 hpu_model_runner.py:1473] [Warmup][Prompt][12/31] batch_size:16 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:36 hpu_model_runner.py:1473] [Warmup][Prompt][13/31] batch_size:2 seq_len:896 free_mem:22.86 GiB
INFO 01-31 03:21:36 hpu_model_runner.py:1473] [Warmup][Prompt][13/31] batch_size:2 seq_len:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:37 hpu_model_runner.py:1473] [Warmup][Prompt][14/31] batch_size:2 seq_len:768 free_mem:22.86 GiB
INFO 01-31 03:21:37 hpu_model_runner.py:1473] [Warmup][Prompt][14/31] batch_size:2 seq_len:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:38 hpu_model_runner.py:1473] [Warmup][Prompt][15/31] batch_size:4 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:21:38 hpu_model_runner.py:1473] [Warmup][Prompt][15/31] batch_size:4 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:39 hpu_model_runner.py:1473] [Warmup][Prompt][16/31] batch_size:2 seq_len:640 free_mem:22.86 GiB
INFO 01-31 03:21:39 hpu_model_runner.py:1473] [Warmup][Prompt][16/31] batch_size:2 seq_len:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:40 hpu_model_runner.py:1473] [Warmup][Prompt][17/31] batch_size:1 seq_len:1024 free_mem:22.86 GiB
INFO 01-31 03:21:40 hpu_model_runner.py:1473] [Warmup][Prompt][17/31] batch_size:1 seq_len:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:41 hpu_model_runner.py:1473] [Warmup][Prompt][18/31] batch_size:2 seq_len:512 free_mem:22.86 GiB
INFO 01-31 03:21:41 hpu_model_runner.py:1473] [Warmup][Prompt][18/31] batch_size:2 seq_len:512 free_mem:22.86 GiB
INFO 01-31 03:21:42 hpu_model_runner.py:1473] [Warmup][Prompt][19/31] batch_size:4 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:42 hpu_model_runner.py:1473] [Warmup][Prompt][19/31] batch_size:4 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:43 hpu_model_runner.py:1473] [Warmup][Prompt][20/31] batch_size:8 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:21:43 hpu_model_runner.py:1473] [Warmup][Prompt][20/31] batch_size:8 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:44 hpu_model_runner.py:1473] [Warmup][Prompt][21/31] batch_size:1 seq_len:896 free_mem:22.86 GiB
INFO 01-31 03:21:44 hpu_model_runner.py:1473] [Warmup][Prompt][21/31] batch_size:1 seq_len:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:45 hpu_model_runner.py:1473] [Warmup][Prompt][22/31] batch_size:1 seq_len:768 free_mem:22.86 GiB
INFO 01-31 03:21:45 hpu_model_runner.py:1473] [Warmup][Prompt][22/31] batch_size:1 seq_len:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:46 hpu_model_runner.py:1473] [Warmup][Prompt][23/31] batch_size:2 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:21:46 hpu_model_runner.py:1473] [Warmup][Prompt][23/31] batch_size:2 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:46 hpu_model_runner.py:1473] [Warmup][Prompt][24/31] batch_size:1 seq_len:640 free_mem:22.86 GiB
INFO 01-31 03:21:46 hpu_model_runner.py:1473] [Warmup][Prompt][24/31] batch_size:1 seq_len:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:47 hpu_model_runner.py:1473] [Warmup][Prompt][25/31] batch_size:1 seq_len:512 free_mem:22.86 GiB
INFO 01-31 03:21:47 hpu_model_runner.py:1473] [Warmup][Prompt][25/31] batch_size:1 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:48 hpu_model_runner.py:1473] [Warmup][Prompt][26/31] batch_size:2 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:21:48 hpu_model_runner.py:1473] [Warmup][Prompt][26/31] batch_size:2 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:49 hpu_model_runner.py:1473] [Warmup][Prompt][27/31] batch_size:4 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:21:49 hpu_model_runner.py:1473] [Warmup][Prompt][27/31] batch_size:4 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:50 hpu_model_runner.py:1473] [Warmup][Prompt][28/31] batch_size:1 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:21:50 hpu_model_runner.py:1473] [Warmup][Prompt][28/31] batch_size:1 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:51 hpu_model_runner.py:1473] [Warmup][Prompt][29/31] batch_size:1 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:21:51 hpu_model_runner.py:1473] [Warmup][Prompt][29/31] batch_size:1 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:51 hpu_model_runner.py:1473] [Warmup][Prompt][30/31] batch_size:2 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:21:51 hpu_model_runner.py:1473] [Warmup][Prompt][30/31] batch_size:2 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:52 hpu_model_runner.py:1473] [Warmup][Prompt][31/31] batch_size:1 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:21:52 hpu_model_runner.py:1473] [Warmup][Prompt][31/31] batch_size:1 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:53 hpu_model_runner.py:1473] [Warmup][Decode][1/325] batch_size:256 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:21:53 hpu_model_runner.py:1473] [Warmup][Decode][1/325] batch_size:256 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:21:56 hpu_model_runner.py:1473] [Warmup][Decode][2/325] batch_size:256 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:21:57 hpu_model_runner.py:1473] [Warmup][Decode][2/325] batch_size:256 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:00 hpu_model_runner.py:1473] [Warmup][Decode][3/325] batch_size:256 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:22:00 hpu_model_runner.py:1473] [Warmup][Decode][3/325] batch_size:256 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:22:03 hpu_model_runner.py:1473] [Warmup][Decode][4/325] batch_size:256 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:03 hpu_model_runner.py:1473] [Warmup][Decode][4/325] batch_size:256 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:22:06 hpu_model_runner.py:1473] [Warmup][Decode][5/325] batch_size:224 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:06 hpu_model_runner.py:1473] [Warmup][Decode][5/325] batch_size:224 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:10 hpu_model_runner.py:1473] [Warmup][Decode][6/325] batch_size:224 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:22:10 hpu_model_runner.py:1473] [Warmup][Decode][6/325] batch_size:224 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:13 hpu_model_runner.py:1473] [Warmup][Decode][7/325] batch_size:256 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:22:13 hpu_model_runner.py:1473] [Warmup][Decode][7/325] batch_size:256 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:22:16 hpu_model_runner.py:1473] [Warmup][Decode][8/325] batch_size:224 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:16 hpu_model_runner.py:1473] [Warmup][Decode][8/325] batch_size:224 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:22:20 hpu_model_runner.py:1473] [Warmup][Decode][9/325] batch_size:256 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:20 hpu_model_runner.py:1473] [Warmup][Decode][9/325] batch_size:256 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:22:23 hpu_model_runner.py:1473] [Warmup][Decode][10/325] batch_size:224 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:23 hpu_model_runner.py:1473] [Warmup][Decode][10/325] batch_size:224 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:22:26 hpu_model_runner.py:1473] [Warmup][Decode][11/325] batch_size:256 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:26 hpu_model_runner.py:1473] [Warmup][Decode][11/325] batch_size:256 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:22:29 hpu_model_runner.py:1473] [Warmup][Decode][12/325] batch_size:224 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:29 hpu_model_runner.py:1473] [Warmup][Decode][12/325] batch_size:224 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:22:32 hpu_model_runner.py:1473] [Warmup][Decode][13/325] batch_size:192 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:32 hpu_model_runner.py:1473] [Warmup][Decode][13/325] batch_size:192 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:36 hpu_model_runner.py:1473] [Warmup][Decode][14/325] batch_size:192 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:22:36 hpu_model_runner.py:1473] [Warmup][Decode][14/325] batch_size:192 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:39 hpu_model_runner.py:1473] [Warmup][Decode][15/325] batch_size:256 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:22:39 hpu_model_runner.py:1473] [Warmup][Decode][15/325] batch_size:256 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:22:42 hpu_model_runner.py:1473] [Warmup][Decode][16/325] batch_size:224 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:42 hpu_model_runner.py:1473] [Warmup][Decode][16/325] batch_size:224 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:45 hpu_model_runner.py:1473] [Warmup][Decode][17/325] batch_size:192 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:22:45 hpu_model_runner.py:1473] [Warmup][Decode][17/325] batch_size:192 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:22:48 hpu_model_runner.py:1473] [Warmup][Decode][18/325] batch_size:256 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:48 hpu_model_runner.py:1473] [Warmup][Decode][18/325] batch_size:256 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:22:51 hpu_model_runner.py:1473] [Warmup][Decode][19/325] batch_size:224 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:51 hpu_model_runner.py:1473] [Warmup][Decode][19/325] batch_size:224 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:22:54 hpu_model_runner.py:1473] [Warmup][Decode][20/325] batch_size:192 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:54 hpu_model_runner.py:1473] [Warmup][Decode][20/325] batch_size:192 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:22:57 hpu_model_runner.py:1473] [Warmup][Decode][21/325] batch_size:256 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:57 hpu_model_runner.py:1473] [Warmup][Decode][21/325] batch_size:256 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:22:59 hpu_model_runner.py:1473] [Warmup][Decode][22/325] batch_size:192 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:22:59 hpu_model_runner.py:1473] [Warmup][Decode][22/325] batch_size:192 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:23:02 hpu_model_runner.py:1473] [Warmup][Decode][23/325] batch_size:224 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:03 hpu_model_runner.py:1473] [Warmup][Decode][23/325] batch_size:224 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:23:05 hpu_model_runner.py:1473] [Warmup][Decode][24/325] batch_size:160 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:05 hpu_model_runner.py:1473] [Warmup][Decode][24/325] batch_size:160 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:09 hpu_model_runner.py:1473] [Warmup][Decode][25/325] batch_size:160 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:23:09 hpu_model_runner.py:1473] [Warmup][Decode][25/325] batch_size:160 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:12 hpu_model_runner.py:1473] [Warmup][Decode][26/325] batch_size:192 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:23:12 hpu_model_runner.py:1473] [Warmup][Decode][26/325] batch_size:192 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:15 hpu_model_runner.py:1473] [Warmup][Decode][27/325] batch_size:256 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:23:15 hpu_model_runner.py:1473] [Warmup][Decode][27/325] batch_size:256 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:18 hpu_model_runner.py:1473] [Warmup][Decode][28/325] batch_size:224 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:23:18 hpu_model_runner.py:1473] [Warmup][Decode][28/325] batch_size:224 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:21 hpu_model_runner.py:1473] [Warmup][Decode][29/325] batch_size:160 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:23:21 hpu_model_runner.py:1473] [Warmup][Decode][29/325] batch_size:160 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:24 hpu_model_runner.py:1473] [Warmup][Decode][30/325] batch_size:192 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:23:24 hpu_model_runner.py:1473] [Warmup][Decode][30/325] batch_size:192 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:27 hpu_model_runner.py:1473] [Warmup][Decode][31/325] batch_size:224 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:23:27 hpu_model_runner.py:1473] [Warmup][Decode][31/325] batch_size:224 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:23:29 hpu_model_runner.py:1473] [Warmup][Decode][32/325] batch_size:256 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:29 hpu_model_runner.py:1473] [Warmup][Decode][32/325] batch_size:256 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:32 hpu_model_runner.py:1473] [Warmup][Decode][33/325] batch_size:160 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:23:32 hpu_model_runner.py:1473] [Warmup][Decode][33/325] batch_size:160 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:23:35 hpu_model_runner.py:1473] [Warmup][Decode][34/325] batch_size:192 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:35 hpu_model_runner.py:1473] [Warmup][Decode][34/325] batch_size:192 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:23:38 hpu_model_runner.py:1473] [Warmup][Decode][35/325] batch_size:160 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:38 hpu_model_runner.py:1473] [Warmup][Decode][35/325] batch_size:160 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:23:41 hpu_model_runner.py:1473] [Warmup][Decode][36/325] batch_size:224 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:41 hpu_model_runner.py:1473] [Warmup][Decode][36/325] batch_size:224 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:23:43 hpu_model_runner.py:1473] [Warmup][Decode][37/325] batch_size:256 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:43 hpu_model_runner.py:1473] [Warmup][Decode][37/325] batch_size:256 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:23:46 hpu_model_runner.py:1473] [Warmup][Decode][38/325] batch_size:192 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:46 hpu_model_runner.py:1473] [Warmup][Decode][38/325] batch_size:192 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:48 hpu_model_runner.py:1473] [Warmup][Decode][39/325] batch_size:160 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:23:48 hpu_model_runner.py:1473] [Warmup][Decode][39/325] batch_size:160 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:51 hpu_model_runner.py:1473] [Warmup][Decode][40/325] batch_size:224 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:23:52 hpu_model_runner.py:1473] [Warmup][Decode][40/325] batch_size:224 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:54 hpu_model_runner.py:1473] [Warmup][Decode][41/325] batch_size:128 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:23:54 hpu_model_runner.py:1473] [Warmup][Decode][41/325] batch_size:128 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:23:58 hpu_model_runner.py:1473] [Warmup][Decode][42/325] batch_size:128 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:23:58 hpu_model_runner.py:1473] [Warmup][Decode][42/325] batch_size:128 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:01 hpu_model_runner.py:1473] [Warmup][Decode][43/325] batch_size:192 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:24:01 hpu_model_runner.py:1473] [Warmup][Decode][43/325] batch_size:192 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:24:04 hpu_model_runner.py:1473] [Warmup][Decode][44/325] batch_size:256 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:04 hpu_model_runner.py:1473] [Warmup][Decode][44/325] batch_size:256 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:24:06 hpu_model_runner.py:1473] [Warmup][Decode][45/325] batch_size:160 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:06 hpu_model_runner.py:1473] [Warmup][Decode][45/325] batch_size:160 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:24:09 hpu_model_runner.py:1473] [Warmup][Decode][46/325] batch_size:128 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:09 hpu_model_runner.py:1473] [Warmup][Decode][46/325] batch_size:128 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:24:12 hpu_model_runner.py:1473] [Warmup][Decode][47/325] batch_size:224 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:12 hpu_model_runner.py:1473] [Warmup][Decode][47/325] batch_size:224 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:24:15 hpu_model_runner.py:1473] [Warmup][Decode][48/325] batch_size:160 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:15 hpu_model_runner.py:1473] [Warmup][Decode][48/325] batch_size:160 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:24:17 hpu_model_runner.py:1473] [Warmup][Decode][49/325] batch_size:192 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:17 hpu_model_runner.py:1473] [Warmup][Decode][49/325] batch_size:192 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:24:20 hpu_model_runner.py:1473] [Warmup][Decode][50/325] batch_size:128 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:20 hpu_model_runner.py:1473] [Warmup][Decode][50/325] batch_size:128 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:24:23 hpu_model_runner.py:1473] [Warmup][Decode][51/325] batch_size:256 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:23 hpu_model_runner.py:1473] [Warmup][Decode][51/325] batch_size:256 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:24:25 hpu_model_runner.py:1473] [Warmup][Decode][52/325] batch_size:160 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:25 hpu_model_runner.py:1473] [Warmup][Decode][52/325] batch_size:160 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:24:28 hpu_model_runner.py:1473] [Warmup][Decode][53/325] batch_size:128 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:28 hpu_model_runner.py:1473] [Warmup][Decode][53/325] batch_size:128 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:31 hpu_model_runner.py:1473] [Warmup][Decode][54/325] batch_size:192 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:24:31 hpu_model_runner.py:1473] [Warmup][Decode][54/325] batch_size:192 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:24:34 hpu_model_runner.py:1473] [Warmup][Decode][55/325] batch_size:224 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:34 hpu_model_runner.py:1473] [Warmup][Decode][55/325] batch_size:224 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:24:36 hpu_model_runner.py:1473] [Warmup][Decode][56/325] batch_size:128 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:36 hpu_model_runner.py:1473] [Warmup][Decode][56/325] batch_size:128 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:24:39 hpu_model_runner.py:1473] [Warmup][Decode][57/325] batch_size:160 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:39 hpu_model_runner.py:1473] [Warmup][Decode][57/325] batch_size:160 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:24:41 hpu_model_runner.py:1473] [Warmup][Decode][58/325] batch_size:256 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:42 hpu_model_runner.py:1473] [Warmup][Decode][58/325] batch_size:256 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:24:44 hpu_model_runner.py:1473] [Warmup][Decode][59/325] batch_size:192 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:44 hpu_model_runner.py:1473] [Warmup][Decode][59/325] batch_size:192 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:24:46 hpu_model_runner.py:1473] [Warmup][Decode][60/325] batch_size:224 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:46 hpu_model_runner.py:1473] [Warmup][Decode][60/325] batch_size:224 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:24:48 hpu_model_runner.py:1473] [Warmup][Decode][61/325] batch_size:128 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:48 hpu_model_runner.py:1473] [Warmup][Decode][61/325] batch_size:128 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:24:51 hpu_model_runner.py:1473] [Warmup][Decode][62/325] batch_size:160 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:51 hpu_model_runner.py:1473] [Warmup][Decode][62/325] batch_size:160 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:24:54 hpu_model_runner.py:1473] [Warmup][Decode][63/325] batch_size:96 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:54 hpu_model_runner.py:1473] [Warmup][Decode][63/325] batch_size:96 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:24:57 hpu_model_runner.py:1473] [Warmup][Decode][64/325] batch_size:96 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:24:57 hpu_model_runner.py:1473] [Warmup][Decode][64/325] batch_size:96 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:25:01 hpu_model_runner.py:1473] [Warmup][Decode][65/325] batch_size:128 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:01 hpu_model_runner.py:1473] [Warmup][Decode][65/325] batch_size:128 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:25:03 hpu_model_runner.py:1473] [Warmup][Decode][66/325] batch_size:192 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:03 hpu_model_runner.py:1473] [Warmup][Decode][66/325] batch_size:192 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:25:06 hpu_model_runner.py:1473] [Warmup][Decode][67/325] batch_size:256 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:06 hpu_model_runner.py:1473] [Warmup][Decode][67/325] batch_size:256 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:25:08 hpu_model_runner.py:1473] [Warmup][Decode][68/325] batch_size:160 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:08 hpu_model_runner.py:1473] [Warmup][Decode][68/325] batch_size:160 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:25:10 hpu_model_runner.py:1473] [Warmup][Decode][69/325] batch_size:224 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:10 hpu_model_runner.py:1473] [Warmup][Decode][69/325] batch_size:224 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:12 hpu_model_runner.py:1473] [Warmup][Decode][70/325] batch_size:96 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:25:12 hpu_model_runner.py:1473] [Warmup][Decode][70/325] batch_size:96 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:25:15 hpu_model_runner.py:1473] [Warmup][Decode][71/325] batch_size:128 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:16 hpu_model_runner.py:1473] [Warmup][Decode][71/325] batch_size:128 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:25:18 hpu_model_runner.py:1473] [Warmup][Decode][72/325] batch_size:96 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:18 hpu_model_runner.py:1473] [Warmup][Decode][72/325] batch_size:96 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:25:21 hpu_model_runner.py:1473] [Warmup][Decode][73/325] batch_size:192 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:21 hpu_model_runner.py:1473] [Warmup][Decode][73/325] batch_size:192 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:25:24 hpu_model_runner.py:1473] [Warmup][Decode][74/325] batch_size:160 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:24 hpu_model_runner.py:1473] [Warmup][Decode][74/325] batch_size:160 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:25:26 hpu_model_runner.py:1473] [Warmup][Decode][75/325] batch_size:128 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:26 hpu_model_runner.py:1473] [Warmup][Decode][75/325] batch_size:128 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:25:29 hpu_model_runner.py:1473] [Warmup][Decode][76/325] batch_size:256 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:29 hpu_model_runner.py:1473] [Warmup][Decode][76/325] batch_size:256 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:25:30 hpu_model_runner.py:1473] [Warmup][Decode][77/325] batch_size:96 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:30 hpu_model_runner.py:1473] [Warmup][Decode][77/325] batch_size:96 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:25:34 hpu_model_runner.py:1473] [Warmup][Decode][78/325] batch_size:224 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:34 hpu_model_runner.py:1473] [Warmup][Decode][78/325] batch_size:224 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:25:36 hpu_model_runner.py:1473] [Warmup][Decode][79/325] batch_size:96 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:36 hpu_model_runner.py:1473] [Warmup][Decode][79/325] batch_size:96 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:25:39 hpu_model_runner.py:1473] [Warmup][Decode][80/325] batch_size:128 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:39 hpu_model_runner.py:1473] [Warmup][Decode][80/325] batch_size:128 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:25:41 hpu_model_runner.py:1473] [Warmup][Decode][81/325] batch_size:160 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:41 hpu_model_runner.py:1473] [Warmup][Decode][81/325] batch_size:160 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:25:44 hpu_model_runner.py:1473] [Warmup][Decode][82/325] batch_size:192 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:44 hpu_model_runner.py:1473] [Warmup][Decode][82/325] batch_size:192 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:25:46 hpu_model_runner.py:1473] [Warmup][Decode][83/325] batch_size:96 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:46 hpu_model_runner.py:1473] [Warmup][Decode][83/325] batch_size:96 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:25:49 hpu_model_runner.py:1473] [Warmup][Decode][84/325] batch_size:128 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:49 hpu_model_runner.py:1473] [Warmup][Decode][84/325] batch_size:128 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:51 hpu_model_runner.py:1473] [Warmup][Decode][85/325] batch_size:224 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:25:51 hpu_model_runner.py:1473] [Warmup][Decode][85/325] batch_size:224 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:25:53 hpu_model_runner.py:1473] [Warmup][Decode][86/325] batch_size:256 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:53 hpu_model_runner.py:1473] [Warmup][Decode][86/325] batch_size:256 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:25:55 hpu_model_runner.py:1473] [Warmup][Decode][87/325] batch_size:160 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:55 hpu_model_runner.py:1473] [Warmup][Decode][87/325] batch_size:160 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:25:57 hpu_model_runner.py:1473] [Warmup][Decode][88/325] batch_size:96 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:25:57 hpu_model_runner.py:1473] [Warmup][Decode][88/325] batch_size:96 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:26:00 hpu_model_runner.py:1473] [Warmup][Decode][89/325] batch_size:192 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:00 hpu_model_runner.py:1473] [Warmup][Decode][89/325] batch_size:192 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:26:02 hpu_model_runner.py:1473] [Warmup][Decode][90/325] batch_size:128 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:02 hpu_model_runner.py:1473] [Warmup][Decode][90/325] batch_size:128 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:26:04 hpu_model_runner.py:1473] [Warmup][Decode][91/325] batch_size:96 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:04 hpu_model_runner.py:1473] [Warmup][Decode][91/325] batch_size:96 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:26:07 hpu_model_runner.py:1473] [Warmup][Decode][92/325] batch_size:160 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:07 hpu_model_runner.py:1473] [Warmup][Decode][92/325] batch_size:160 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:26:09 hpu_model_runner.py:1473] [Warmup][Decode][93/325] batch_size:224 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:09 hpu_model_runner.py:1473] [Warmup][Decode][93/325] batch_size:224 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:26:11 hpu_model_runner.py:1473] [Warmup][Decode][94/325] batch_size:64 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:11 hpu_model_runner.py:1473] [Warmup][Decode][94/325] batch_size:64 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:14 hpu_model_runner.py:1473] [Warmup][Decode][95/325] batch_size:64 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:26:15 hpu_model_runner.py:1473] [Warmup][Decode][95/325] batch_size:64 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:26:18 hpu_model_runner.py:1473] [Warmup][Decode][96/325] batch_size:96 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:18 hpu_model_runner.py:1473] [Warmup][Decode][96/325] batch_size:96 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:26:20 hpu_model_runner.py:1473] [Warmup][Decode][97/325] batch_size:128 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:20 hpu_model_runner.py:1473] [Warmup][Decode][97/325] batch_size:128 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:26:23 hpu_model_runner.py:1473] [Warmup][Decode][98/325] batch_size:192 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:23 hpu_model_runner.py:1473] [Warmup][Decode][98/325] batch_size:192 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:26:24 hpu_model_runner.py:1473] [Warmup][Decode][99/325] batch_size:256 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:25 hpu_model_runner.py:1473] [Warmup][Decode][99/325] batch_size:256 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:26:26 hpu_model_runner.py:1473] [Warmup][Decode][100/325] batch_size:64 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:26 hpu_model_runner.py:1473] [Warmup][Decode][100/325] batch_size:64 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:26:29 hpu_model_runner.py:1473] [Warmup][Decode][101/325] batch_size:96 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:29 hpu_model_runner.py:1473] [Warmup][Decode][101/325] batch_size:96 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:26:32 hpu_model_runner.py:1473] [Warmup][Decode][102/325] batch_size:160 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:32 hpu_model_runner.py:1473] [Warmup][Decode][102/325] batch_size:160 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:26:34 hpu_model_runner.py:1473] [Warmup][Decode][103/325] batch_size:64 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:34 hpu_model_runner.py:1473] [Warmup][Decode][103/325] batch_size:64 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:37 hpu_model_runner.py:1473] [Warmup][Decode][104/325] batch_size:128 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:26:37 hpu_model_runner.py:1473] [Warmup][Decode][104/325] batch_size:128 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:26:39 hpu_model_runner.py:1473] [Warmup][Decode][105/325] batch_size:64 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:39 hpu_model_runner.py:1473] [Warmup][Decode][105/325] batch_size:64 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:26:42 hpu_model_runner.py:1473] [Warmup][Decode][106/325] batch_size:96 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:42 hpu_model_runner.py:1473] [Warmup][Decode][106/325] batch_size:96 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:26:45 hpu_model_runner.py:1473] [Warmup][Decode][107/325] batch_size:192 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:45 hpu_model_runner.py:1473] [Warmup][Decode][107/325] batch_size:192 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:26:47 hpu_model_runner.py:1473] [Warmup][Decode][108/325] batch_size:224 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:47 hpu_model_runner.py:1473] [Warmup][Decode][108/325] batch_size:224 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:26:48 hpu_model_runner.py:1473] [Warmup][Decode][109/325] batch_size:64 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:48 hpu_model_runner.py:1473] [Warmup][Decode][109/325] batch_size:64 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:26:51 hpu_model_runner.py:1473] [Warmup][Decode][110/325] batch_size:128 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:51 hpu_model_runner.py:1473] [Warmup][Decode][110/325] batch_size:128 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:26:53 hpu_model_runner.py:1473] [Warmup][Decode][111/325] batch_size:160 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:54 hpu_model_runner.py:1473] [Warmup][Decode][111/325] batch_size:160 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:26:55 hpu_model_runner.py:1473] [Warmup][Decode][112/325] batch_size:256 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:55 hpu_model_runner.py:1473] [Warmup][Decode][112/325] batch_size:256 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:26:57 hpu_model_runner.py:1473] [Warmup][Decode][113/325] batch_size:96 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:57 hpu_model_runner.py:1473] [Warmup][Decode][113/325] batch_size:96 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:26:59 hpu_model_runner.py:1473] [Warmup][Decode][114/325] batch_size:64 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:26:59 hpu_model_runner.py:1473] [Warmup][Decode][114/325] batch_size:64 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:27:02 hpu_model_runner.py:1473] [Warmup][Decode][115/325] batch_size:64 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:02 hpu_model_runner.py:1473] [Warmup][Decode][115/325] batch_size:64 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:27:05 hpu_model_runner.py:1473] [Warmup][Decode][116/325] batch_size:96 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:05 hpu_model_runner.py:1473] [Warmup][Decode][116/325] batch_size:96 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:27:08 hpu_model_runner.py:1473] [Warmup][Decode][117/325] batch_size:128 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:08 hpu_model_runner.py:1473] [Warmup][Decode][117/325] batch_size:128 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:27:10 hpu_model_runner.py:1473] [Warmup][Decode][118/325] batch_size:192 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:10 hpu_model_runner.py:1473] [Warmup][Decode][118/325] batch_size:192 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:27:11 hpu_model_runner.py:1473] [Warmup][Decode][119/325] batch_size:160 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:11 hpu_model_runner.py:1473] [Warmup][Decode][119/325] batch_size:160 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:27:13 hpu_model_runner.py:1473] [Warmup][Decode][120/325] batch_size:224 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:13 hpu_model_runner.py:1473] [Warmup][Decode][120/325] batch_size:224 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:27:15 hpu_model_runner.py:1473] [Warmup][Decode][121/325] batch_size:64 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:15 hpu_model_runner.py:1473] [Warmup][Decode][121/325] batch_size:64 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:27:17 hpu_model_runner.py:1473] [Warmup][Decode][122/325] batch_size:96 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:17 hpu_model_runner.py:1473] [Warmup][Decode][122/325] batch_size:96 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:20 hpu_model_runner.py:1473] [Warmup][Decode][123/325] batch_size:64 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:27:20 hpu_model_runner.py:1473] [Warmup][Decode][123/325] batch_size:64 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:27:22 hpu_model_runner.py:1473] [Warmup][Decode][124/325] batch_size:128 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:22 hpu_model_runner.py:1473] [Warmup][Decode][124/325] batch_size:128 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:27:24 hpu_model_runner.py:1473] [Warmup][Decode][125/325] batch_size:256 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:24 hpu_model_runner.py:1473] [Warmup][Decode][125/325] batch_size:256 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:27:26 hpu_model_runner.py:1473] [Warmup][Decode][126/325] batch_size:64 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:26 hpu_model_runner.py:1473] [Warmup][Decode][126/325] batch_size:64 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:27:28 hpu_model_runner.py:1473] [Warmup][Decode][127/325] batch_size:96 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:28 hpu_model_runner.py:1473] [Warmup][Decode][127/325] batch_size:96 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:27:30 hpu_model_runner.py:1473] [Warmup][Decode][128/325] batch_size:160 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:30 hpu_model_runner.py:1473] [Warmup][Decode][128/325] batch_size:160 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:27:32 hpu_model_runner.py:1473] [Warmup][Decode][129/325] batch_size:192 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:32 hpu_model_runner.py:1473] [Warmup][Decode][129/325] batch_size:192 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:27:34 hpu_model_runner.py:1473] [Warmup][Decode][130/325] batch_size:64 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:34 hpu_model_runner.py:1473] [Warmup][Decode][130/325] batch_size:64 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:27:36 hpu_model_runner.py:1473] [Warmup][Decode][131/325] batch_size:128 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:36 hpu_model_runner.py:1473] [Warmup][Decode][131/325] batch_size:128 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:27:38 hpu_model_runner.py:1473] [Warmup][Decode][132/325] batch_size:224 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:38 hpu_model_runner.py:1473] [Warmup][Decode][132/325] batch_size:224 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:27:39 hpu_model_runner.py:1473] [Warmup][Decode][133/325] batch_size:96 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:39 hpu_model_runner.py:1473] [Warmup][Decode][133/325] batch_size:96 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:27:41 hpu_model_runner.py:1473] [Warmup][Decode][134/325] batch_size:64 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:41 hpu_model_runner.py:1473] [Warmup][Decode][134/325] batch_size:64 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:27:44 hpu_model_runner.py:1473] [Warmup][Decode][135/325] batch_size:160 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:44 hpu_model_runner.py:1473] [Warmup][Decode][135/325] batch_size:160 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:27:45 hpu_model_runner.py:1473] [Warmup][Decode][136/325] batch_size:32 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:45 hpu_model_runner.py:1473] [Warmup][Decode][136/325] batch_size:32 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:27:49 hpu_model_runner.py:1473] [Warmup][Decode][137/325] batch_size:32 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:49 hpu_model_runner.py:1473] [Warmup][Decode][137/325] batch_size:32 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:27:52 hpu_model_runner.py:1473] [Warmup][Decode][138/325] batch_size:64 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:52 hpu_model_runner.py:1473] [Warmup][Decode][138/325] batch_size:64 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:27:54 hpu_model_runner.py:1473] [Warmup][Decode][139/325] batch_size:96 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:54 hpu_model_runner.py:1473] [Warmup][Decode][139/325] batch_size:96 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:56 hpu_model_runner.py:1473] [Warmup][Decode][140/325] batch_size:128 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:27:56 hpu_model_runner.py:1473] [Warmup][Decode][140/325] batch_size:128 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:27:58 hpu_model_runner.py:1473] [Warmup][Decode][141/325] batch_size:192 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:27:58 hpu_model_runner.py:1473] [Warmup][Decode][141/325] batch_size:192 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:00 hpu_model_runner.py:1473] [Warmup][Decode][142/325] batch_size:256 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:28:00 hpu_model_runner.py:1473] [Warmup][Decode][142/325] batch_size:256 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:01 hpu_model_runner.py:1473] [Warmup][Decode][143/325] batch_size:32 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:28:01 hpu_model_runner.py:1473] [Warmup][Decode][143/325] batch_size:32 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:28:04 hpu_model_runner.py:1473] [Warmup][Decode][144/325] batch_size:32 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:04 hpu_model_runner.py:1473] [Warmup][Decode][144/325] batch_size:32 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:28:07 hpu_model_runner.py:1473] [Warmup][Decode][145/325] batch_size:64 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:07 hpu_model_runner.py:1473] [Warmup][Decode][145/325] batch_size:64 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:28:10 hpu_model_runner.py:1473] [Warmup][Decode][146/325] batch_size:32 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:10 hpu_model_runner.py:1473] [Warmup][Decode][146/325] batch_size:32 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:28:13 hpu_model_runner.py:1473] [Warmup][Decode][147/325] batch_size:96 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:13 hpu_model_runner.py:1473] [Warmup][Decode][147/325] batch_size:96 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:28:15 hpu_model_runner.py:1473] [Warmup][Decode][148/325] batch_size:224 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:15 hpu_model_runner.py:1473] [Warmup][Decode][148/325] batch_size:224 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:28:16 hpu_model_runner.py:1473] [Warmup][Decode][149/325] batch_size:32 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:16 hpu_model_runner.py:1473] [Warmup][Decode][149/325] batch_size:32 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:28:19 hpu_model_runner.py:1473] [Warmup][Decode][150/325] batch_size:64 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:19 hpu_model_runner.py:1473] [Warmup][Decode][150/325] batch_size:64 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:28:21 hpu_model_runner.py:1473] [Warmup][Decode][151/325] batch_size:128 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:21 hpu_model_runner.py:1473] [Warmup][Decode][151/325] batch_size:128 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:28:23 hpu_model_runner.py:1473] [Warmup][Decode][152/325] batch_size:160 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:23 hpu_model_runner.py:1473] [Warmup][Decode][152/325] batch_size:160 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:28:24 hpu_model_runner.py:1473] [Warmup][Decode][153/325] batch_size:32 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:24 hpu_model_runner.py:1473] [Warmup][Decode][153/325] batch_size:32 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:28:27 hpu_model_runner.py:1473] [Warmup][Decode][154/325] batch_size:32 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:27 hpu_model_runner.py:1473] [Warmup][Decode][154/325] batch_size:32 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:28:30 hpu_model_runner.py:1473] [Warmup][Decode][155/325] batch_size:64 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:30 hpu_model_runner.py:1473] [Warmup][Decode][155/325] batch_size:64 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:28:32 hpu_model_runner.py:1473] [Warmup][Decode][156/325] batch_size:96 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:32 hpu_model_runner.py:1473] [Warmup][Decode][156/325] batch_size:96 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:28:34 hpu_model_runner.py:1473] [Warmup][Decode][157/325] batch_size:192 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:34 hpu_model_runner.py:1473] [Warmup][Decode][157/325] batch_size:192 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:28:35 hpu_model_runner.py:1473] [Warmup][Decode][158/325] batch_size:32 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:35 hpu_model_runner.py:1473] [Warmup][Decode][158/325] batch_size:32 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:28:38 hpu_model_runner.py:1473] [Warmup][Decode][159/325] batch_size:32 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:38 hpu_model_runner.py:1473] [Warmup][Decode][159/325] batch_size:32 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:41 hpu_model_runner.py:1473] [Warmup][Decode][160/325] batch_size:64 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:28:41 hpu_model_runner.py:1473] [Warmup][Decode][160/325] batch_size:64 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:43 hpu_model_runner.py:1473] [Warmup][Decode][161/325] batch_size:128 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:28:43 hpu_model_runner.py:1473] [Warmup][Decode][161/325] batch_size:128 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:28:44 hpu_model_runner.py:1473] [Warmup][Decode][162/325] batch_size:256 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:44 hpu_model_runner.py:1473] [Warmup][Decode][162/325] batch_size:256 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:28:46 hpu_model_runner.py:1473] [Warmup][Decode][163/325] batch_size:32 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:46 hpu_model_runner.py:1473] [Warmup][Decode][163/325] batch_size:32 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:28:48 hpu_model_runner.py:1473] [Warmup][Decode][164/325] batch_size:96 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:48 hpu_model_runner.py:1473] [Warmup][Decode][164/325] batch_size:96 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:28:50 hpu_model_runner.py:1473] [Warmup][Decode][165/325] batch_size:160 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:50 hpu_model_runner.py:1473] [Warmup][Decode][165/325] batch_size:160 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:28:51 hpu_model_runner.py:1473] [Warmup][Decode][166/325] batch_size:32 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:51 hpu_model_runner.py:1473] [Warmup][Decode][166/325] batch_size:32 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:28:54 hpu_model_runner.py:1473] [Warmup][Decode][167/325] batch_size:64 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:54 hpu_model_runner.py:1473] [Warmup][Decode][167/325] batch_size:64 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:28:56 hpu_model_runner.py:1473] [Warmup][Decode][168/325] batch_size:224 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:56 hpu_model_runner.py:1473] [Warmup][Decode][168/325] batch_size:224 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:28:57 hpu_model_runner.py:1473] [Warmup][Decode][169/325] batch_size:32 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:28:57 hpu_model_runner.py:1473] [Warmup][Decode][169/325] batch_size:32 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:29:00 hpu_model_runner.py:1473] [Warmup][Decode][170/325] batch_size:16 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:00 hpu_model_runner.py:1473] [Warmup][Decode][170/325] batch_size:16 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:29:03 hpu_model_runner.py:1473] [Warmup][Decode][171/325] batch_size:16 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:03 hpu_model_runner.py:1473] [Warmup][Decode][171/325] batch_size:16 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:29:06 hpu_model_runner.py:1473] [Warmup][Decode][172/325] batch_size:32 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:06 hpu_model_runner.py:1473] [Warmup][Decode][172/325] batch_size:32 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:29:09 hpu_model_runner.py:1473] [Warmup][Decode][173/325] batch_size:64 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:09 hpu_model_runner.py:1473] [Warmup][Decode][173/325] batch_size:64 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:29:11 hpu_model_runner.py:1473] [Warmup][Decode][174/325] batch_size:96 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:11 hpu_model_runner.py:1473] [Warmup][Decode][174/325] batch_size:96 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:29:12 hpu_model_runner.py:1473] [Warmup][Decode][175/325] batch_size:128 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:12 hpu_model_runner.py:1473] [Warmup][Decode][175/325] batch_size:128 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:29:14 hpu_model_runner.py:1473] [Warmup][Decode][176/325] batch_size:192 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:14 hpu_model_runner.py:1473] [Warmup][Decode][176/325] batch_size:192 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:29:15 hpu_model_runner.py:1473] [Warmup][Decode][177/325] batch_size:16 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:15 hpu_model_runner.py:1473] [Warmup][Decode][177/325] batch_size:16 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:18 hpu_model_runner.py:1473] [Warmup][Decode][178/325] batch_size:16 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:29:18 hpu_model_runner.py:1473] [Warmup][Decode][178/325] batch_size:16 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:22 hpu_model_runner.py:1473] [Warmup][Decode][179/325] batch_size:32 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:29:22 hpu_model_runner.py:1473] [Warmup][Decode][179/325] batch_size:32 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:29:24 hpu_model_runner.py:1473] [Warmup][Decode][180/325] batch_size:16 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:24 hpu_model_runner.py:1473] [Warmup][Decode][180/325] batch_size:16 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:29:27 hpu_model_runner.py:1473] [Warmup][Decode][181/325] batch_size:16 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:27 hpu_model_runner.py:1473] [Warmup][Decode][181/325] batch_size:16 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:29:30 hpu_model_runner.py:1473] [Warmup][Decode][182/325] batch_size:32 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:30 hpu_model_runner.py:1473] [Warmup][Decode][182/325] batch_size:32 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:29:32 hpu_model_runner.py:1473] [Warmup][Decode][183/325] batch_size:64 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:32 hpu_model_runner.py:1473] [Warmup][Decode][183/325] batch_size:64 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:29:34 hpu_model_runner.py:1473] [Warmup][Decode][184/325] batch_size:160 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:34 hpu_model_runner.py:1473] [Warmup][Decode][184/325] batch_size:160 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:29:35 hpu_model_runner.py:1473] [Warmup][Decode][185/325] batch_size:16 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:35 hpu_model_runner.py:1473] [Warmup][Decode][185/325] batch_size:16 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:29:38 hpu_model_runner.py:1473] [Warmup][Decode][186/325] batch_size:16 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:38 hpu_model_runner.py:1473] [Warmup][Decode][186/325] batch_size:16 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:29:41 hpu_model_runner.py:1473] [Warmup][Decode][187/325] batch_size:32 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:41 hpu_model_runner.py:1473] [Warmup][Decode][187/325] batch_size:32 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:29:43 hpu_model_runner.py:1473] [Warmup][Decode][188/325] batch_size:96 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:43 hpu_model_runner.py:1473] [Warmup][Decode][188/325] batch_size:96 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:29:45 hpu_model_runner.py:1473] [Warmup][Decode][189/325] batch_size:16 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:45 hpu_model_runner.py:1473] [Warmup][Decode][189/325] batch_size:16 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:29:47 hpu_model_runner.py:1473] [Warmup][Decode][190/325] batch_size:16 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:47 hpu_model_runner.py:1473] [Warmup][Decode][190/325] batch_size:16 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:29:50 hpu_model_runner.py:1473] [Warmup][Decode][191/325] batch_size:32 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:50 hpu_model_runner.py:1473] [Warmup][Decode][191/325] batch_size:32 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:29:52 hpu_model_runner.py:1473] [Warmup][Decode][192/325] batch_size:64 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:52 hpu_model_runner.py:1473] [Warmup][Decode][192/325] batch_size:64 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:29:54 hpu_model_runner.py:1473] [Warmup][Decode][193/325] batch_size:128 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:54 hpu_model_runner.py:1473] [Warmup][Decode][193/325] batch_size:128 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:29:55 hpu_model_runner.py:1473] [Warmup][Decode][194/325] batch_size:256 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:55 hpu_model_runner.py:1473] [Warmup][Decode][194/325] batch_size:256 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:29:56 hpu_model_runner.py:1473] [Warmup][Decode][195/325] batch_size:16 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:56 hpu_model_runner.py:1473] [Warmup][Decode][195/325] batch_size:16 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:29:59 hpu_model_runner.py:1473] [Warmup][Decode][196/325] batch_size:16 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:29:59 hpu_model_runner.py:1473] [Warmup][Decode][196/325] batch_size:16 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:30:01 hpu_model_runner.py:1473] [Warmup][Decode][197/325] batch_size:32 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:01 hpu_model_runner.py:1473] [Warmup][Decode][197/325] batch_size:32 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:30:03 hpu_model_runner.py:1473] [Warmup][Decode][198/325] batch_size:224 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:03 hpu_model_runner.py:1473] [Warmup][Decode][198/325] batch_size:224 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:05 hpu_model_runner.py:1473] [Warmup][Decode][199/325] batch_size:16 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:30:05 hpu_model_runner.py:1473] [Warmup][Decode][199/325] batch_size:16 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:30:07 hpu_model_runner.py:1473] [Warmup][Decode][200/325] batch_size:8 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:07 hpu_model_runner.py:1473] [Warmup][Decode][200/325] batch_size:8 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:30:11 hpu_model_runner.py:1473] [Warmup][Decode][201/325] batch_size:8 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:11 hpu_model_runner.py:1473] [Warmup][Decode][201/325] batch_size:8 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:30:14 hpu_model_runner.py:1473] [Warmup][Decode][202/325] batch_size:16 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:14 hpu_model_runner.py:1473] [Warmup][Decode][202/325] batch_size:16 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:30:16 hpu_model_runner.py:1473] [Warmup][Decode][203/325] batch_size:32 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:16 hpu_model_runner.py:1473] [Warmup][Decode][203/325] batch_size:32 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:30:18 hpu_model_runner.py:1473] [Warmup][Decode][204/325] batch_size:64 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:18 hpu_model_runner.py:1473] [Warmup][Decode][204/325] batch_size:64 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:30:19 hpu_model_runner.py:1473] [Warmup][Decode][205/325] batch_size:96 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:19 hpu_model_runner.py:1473] [Warmup][Decode][205/325] batch_size:96 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:30:21 hpu_model_runner.py:1473] [Warmup][Decode][206/325] batch_size:192 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:21 hpu_model_runner.py:1473] [Warmup][Decode][206/325] batch_size:192 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:22 hpu_model_runner.py:1473] [Warmup][Decode][207/325] batch_size:8 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:30:22 hpu_model_runner.py:1473] [Warmup][Decode][207/325] batch_size:8 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:30:25 hpu_model_runner.py:1473] [Warmup][Decode][208/325] batch_size:8 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:25 hpu_model_runner.py:1473] [Warmup][Decode][208/325] batch_size:8 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:30:29 hpu_model_runner.py:1473] [Warmup][Decode][209/325] batch_size:16 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:29 hpu_model_runner.py:1473] [Warmup][Decode][209/325] batch_size:16 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:30:31 hpu_model_runner.py:1473] [Warmup][Decode][210/325] batch_size:8 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:31 hpu_model_runner.py:1473] [Warmup][Decode][210/325] batch_size:8 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:30:34 hpu_model_runner.py:1473] [Warmup][Decode][211/325] batch_size:8 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:34 hpu_model_runner.py:1473] [Warmup][Decode][211/325] batch_size:8 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:30:37 hpu_model_runner.py:1473] [Warmup][Decode][212/325] batch_size:16 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:37 hpu_model_runner.py:1473] [Warmup][Decode][212/325] batch_size:16 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:30:39 hpu_model_runner.py:1473] [Warmup][Decode][213/325] batch_size:32 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:39 hpu_model_runner.py:1473] [Warmup][Decode][213/325] batch_size:32 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:30:41 hpu_model_runner.py:1473] [Warmup][Decode][214/325] batch_size:160 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:41 hpu_model_runner.py:1473] [Warmup][Decode][214/325] batch_size:160 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:30:42 hpu_model_runner.py:1473] [Warmup][Decode][215/325] batch_size:8 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:42 hpu_model_runner.py:1473] [Warmup][Decode][215/325] batch_size:8 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:30:45 hpu_model_runner.py:1473] [Warmup][Decode][216/325] batch_size:8 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:45 hpu_model_runner.py:1473] [Warmup][Decode][216/325] batch_size:8 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:30:48 hpu_model_runner.py:1473] [Warmup][Decode][217/325] batch_size:16 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:48 hpu_model_runner.py:1473] [Warmup][Decode][217/325] batch_size:16 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:50 hpu_model_runner.py:1473] [Warmup][Decode][218/325] batch_size:8 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:30:50 hpu_model_runner.py:1473] [Warmup][Decode][218/325] batch_size:8 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:30:53 hpu_model_runner.py:1473] [Warmup][Decode][219/325] batch_size:8 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:53 hpu_model_runner.py:1473] [Warmup][Decode][219/325] batch_size:8 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:30:56 hpu_model_runner.py:1473] [Warmup][Decode][220/325] batch_size:16 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:56 hpu_model_runner.py:1473] [Warmup][Decode][220/325] batch_size:16 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:30:58 hpu_model_runner.py:1473] [Warmup][Decode][221/325] batch_size:32 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:58 hpu_model_runner.py:1473] [Warmup][Decode][221/325] batch_size:32 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:30:59 hpu_model_runner.py:1473] [Warmup][Decode][222/325] batch_size:64 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:30:59 hpu_model_runner.py:1473] [Warmup][Decode][222/325] batch_size:64 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:31:01 hpu_model_runner.py:1473] [Warmup][Decode][223/325] batch_size:128 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:01 hpu_model_runner.py:1473] [Warmup][Decode][223/325] batch_size:128 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:31:02 hpu_model_runner.py:1473] [Warmup][Decode][224/325] batch_size:8 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:02 hpu_model_runner.py:1473] [Warmup][Decode][224/325] batch_size:8 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:31:04 hpu_model_runner.py:1473] [Warmup][Decode][225/325] batch_size:8 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:04 hpu_model_runner.py:1473] [Warmup][Decode][225/325] batch_size:8 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:31:07 hpu_model_runner.py:1473] [Warmup][Decode][226/325] batch_size:16 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:07 hpu_model_runner.py:1473] [Warmup][Decode][226/325] batch_size:16 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:31:09 hpu_model_runner.py:1473] [Warmup][Decode][227/325] batch_size:8 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:09 hpu_model_runner.py:1473] [Warmup][Decode][227/325] batch_size:8 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:31:11 hpu_model_runner.py:1473] [Warmup][Decode][228/325] batch_size:4 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:11 hpu_model_runner.py:1473] [Warmup][Decode][228/325] batch_size:4 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:31:15 hpu_model_runner.py:1473] [Warmup][Decode][229/325] batch_size:4 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:15 hpu_model_runner.py:1473] [Warmup][Decode][229/325] batch_size:4 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:31:18 hpu_model_runner.py:1473] [Warmup][Decode][230/325] batch_size:8 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:18 hpu_model_runner.py:1473] [Warmup][Decode][230/325] batch_size:8 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:31:20 hpu_model_runner.py:1473] [Warmup][Decode][231/325] batch_size:16 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:21 hpu_model_runner.py:1473] [Warmup][Decode][231/325] batch_size:16 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:31:22 hpu_model_runner.py:1473] [Warmup][Decode][232/325] batch_size:32 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:22 hpu_model_runner.py:1473] [Warmup][Decode][232/325] batch_size:32 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:31:24 hpu_model_runner.py:1473] [Warmup][Decode][233/325] batch_size:96 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:24 hpu_model_runner.py:1473] [Warmup][Decode][233/325] batch_size:96 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:31:25 hpu_model_runner.py:1473] [Warmup][Decode][234/325] batch_size:4 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:25 hpu_model_runner.py:1473] [Warmup][Decode][234/325] batch_size:4 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:28 hpu_model_runner.py:1473] [Warmup][Decode][235/325] batch_size:4 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:31:28 hpu_model_runner.py:1473] [Warmup][Decode][235/325] batch_size:4 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:32 hpu_model_runner.py:1473] [Warmup][Decode][236/325] batch_size:8 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:31:32 hpu_model_runner.py:1473] [Warmup][Decode][236/325] batch_size:8 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:31:34 hpu_model_runner.py:1473] [Warmup][Decode][237/325] batch_size:4 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:34 hpu_model_runner.py:1473] [Warmup][Decode][237/325] batch_size:4 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:31:37 hpu_model_runner.py:1473] [Warmup][Decode][238/325] batch_size:4 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:37 hpu_model_runner.py:1473] [Warmup][Decode][238/325] batch_size:4 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:31:40 hpu_model_runner.py:1473] [Warmup][Decode][239/325] batch_size:8 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:40 hpu_model_runner.py:1473] [Warmup][Decode][239/325] batch_size:8 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:31:42 hpu_model_runner.py:1473] [Warmup][Decode][240/325] batch_size:16 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:42 hpu_model_runner.py:1473] [Warmup][Decode][240/325] batch_size:16 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:31:44 hpu_model_runner.py:1473] [Warmup][Decode][241/325] batch_size:4 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:44 hpu_model_runner.py:1473] [Warmup][Decode][241/325] batch_size:4 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:31:47 hpu_model_runner.py:1473] [Warmup][Decode][242/325] batch_size:4 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:47 hpu_model_runner.py:1473] [Warmup][Decode][242/325] batch_size:4 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:31:50 hpu_model_runner.py:1473] [Warmup][Decode][243/325] batch_size:8 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:50 hpu_model_runner.py:1473] [Warmup][Decode][243/325] batch_size:8 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:31:52 hpu_model_runner.py:1473] [Warmup][Decode][244/325] batch_size:4 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:52 hpu_model_runner.py:1473] [Warmup][Decode][244/325] batch_size:4 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:31:54 hpu_model_runner.py:1473] [Warmup][Decode][245/325] batch_size:4 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:55 hpu_model_runner.py:1473] [Warmup][Decode][245/325] batch_size:4 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:31:57 hpu_model_runner.py:1473] [Warmup][Decode][246/325] batch_size:8 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:57 hpu_model_runner.py:1473] [Warmup][Decode][246/325] batch_size:8 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:31:59 hpu_model_runner.py:1473] [Warmup][Decode][247/325] batch_size:16 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:31:59 hpu_model_runner.py:1473] [Warmup][Decode][247/325] batch_size:16 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:32:01 hpu_model_runner.py:1473] [Warmup][Decode][248/325] batch_size:32 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:01 hpu_model_runner.py:1473] [Warmup][Decode][248/325] batch_size:32 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:32:02 hpu_model_runner.py:1473] [Warmup][Decode][249/325] batch_size:64 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:02 hpu_model_runner.py:1473] [Warmup][Decode][249/325] batch_size:64 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:32:04 hpu_model_runner.py:1473] [Warmup][Decode][250/325] batch_size:4 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:04 hpu_model_runner.py:1473] [Warmup][Decode][250/325] batch_size:4 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:32:06 hpu_model_runner.py:1473] [Warmup][Decode][251/325] batch_size:4 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:06 hpu_model_runner.py:1473] [Warmup][Decode][251/325] batch_size:4 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:32:09 hpu_model_runner.py:1473] [Warmup][Decode][252/325] batch_size:8 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:09 hpu_model_runner.py:1473] [Warmup][Decode][252/325] batch_size:8 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:32:10 hpu_model_runner.py:1473] [Warmup][Decode][253/325] batch_size:4 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:10 hpu_model_runner.py:1473] [Warmup][Decode][253/325] batch_size:4 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:13 hpu_model_runner.py:1473] [Warmup][Decode][254/325] batch_size:2 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:32:13 hpu_model_runner.py:1473] [Warmup][Decode][254/325] batch_size:2 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:32:16 hpu_model_runner.py:1473] [Warmup][Decode][255/325] batch_size:2 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:16 hpu_model_runner.py:1473] [Warmup][Decode][255/325] batch_size:2 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:32:20 hpu_model_runner.py:1473] [Warmup][Decode][256/325] batch_size:4 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:20 hpu_model_runner.py:1473] [Warmup][Decode][256/325] batch_size:4 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:32:22 hpu_model_runner.py:1473] [Warmup][Decode][257/325] batch_size:8 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:22 hpu_model_runner.py:1473] [Warmup][Decode][257/325] batch_size:8 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:32:24 hpu_model_runner.py:1473] [Warmup][Decode][258/325] batch_size:16 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:24 hpu_model_runner.py:1473] [Warmup][Decode][258/325] batch_size:16 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:32:25 hpu_model_runner.py:1473] [Warmup][Decode][259/325] batch_size:2 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:25 hpu_model_runner.py:1473] [Warmup][Decode][259/325] batch_size:2 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:32:29 hpu_model_runner.py:1473] [Warmup][Decode][260/325] batch_size:2 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:29 hpu_model_runner.py:1473] [Warmup][Decode][260/325] batch_size:2 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:32:32 hpu_model_runner.py:1473] [Warmup][Decode][261/325] batch_size:4 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:32 hpu_model_runner.py:1473] [Warmup][Decode][261/325] batch_size:4 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:32:34 hpu_model_runner.py:1473] [Warmup][Decode][262/325] batch_size:2 num_blocks:2688 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:34 hpu_model_runner.py:1473] [Warmup][Decode][262/325] batch_size:2 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:32:37 hpu_model_runner.py:1473] [Warmup][Decode][263/325] batch_size:2 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:37 hpu_model_runner.py:1473] [Warmup][Decode][263/325] batch_size:2 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:32:40 hpu_model_runner.py:1473] [Warmup][Decode][264/325] batch_size:4 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:40 hpu_model_runner.py:1473] [Warmup][Decode][264/325] batch_size:4 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:32:42 hpu_model_runner.py:1473] [Warmup][Decode][265/325] batch_size:8 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:42 hpu_model_runner.py:1473] [Warmup][Decode][265/325] batch_size:8 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:32:44 hpu_model_runner.py:1473] [Warmup][Decode][266/325] batch_size:2 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:44 hpu_model_runner.py:1473] [Warmup][Decode][266/325] batch_size:2 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:32:47 hpu_model_runner.py:1473] [Warmup][Decode][267/325] batch_size:2 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:47 hpu_model_runner.py:1473] [Warmup][Decode][267/325] batch_size:2 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:32:50 hpu_model_runner.py:1473] [Warmup][Decode][268/325] batch_size:4 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:50 hpu_model_runner.py:1473] [Warmup][Decode][268/325] batch_size:4 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:32:52 hpu_model_runner.py:1473] [Warmup][Decode][269/325] batch_size:2 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:52 hpu_model_runner.py:1473] [Warmup][Decode][269/325] batch_size:2 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:55 hpu_model_runner.py:1473] [Warmup][Decode][270/325] batch_size:2 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:32:55 hpu_model_runner.py:1473] [Warmup][Decode][270/325] batch_size:2 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:32:57 hpu_model_runner.py:1473] [Warmup][Decode][271/325] batch_size:4 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:57 hpu_model_runner.py:1473] [Warmup][Decode][271/325] batch_size:4 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:32:59 hpu_model_runner.py:1473] [Warmup][Decode][272/325] batch_size:8 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:32:59 hpu_model_runner.py:1473] [Warmup][Decode][272/325] batch_size:8 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:33:01 hpu_model_runner.py:1473] [Warmup][Decode][273/325] batch_size:16 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:01 hpu_model_runner.py:1473] [Warmup][Decode][273/325] batch_size:16 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:33:02 hpu_model_runner.py:1473] [Warmup][Decode][274/325] batch_size:32 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:02 hpu_model_runner.py:1473] [Warmup][Decode][274/325] batch_size:32 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:33:04 hpu_model_runner.py:1473] [Warmup][Decode][275/325] batch_size:2 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:04 hpu_model_runner.py:1473] [Warmup][Decode][275/325] batch_size:2 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:33:06 hpu_model_runner.py:1473] [Warmup][Decode][276/325] batch_size:2 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:06 hpu_model_runner.py:1473] [Warmup][Decode][276/325] batch_size:2 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:33:09 hpu_model_runner.py:1473] [Warmup][Decode][277/325] batch_size:4 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:09 hpu_model_runner.py:1473] [Warmup][Decode][277/325] batch_size:4 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:33:10 hpu_model_runner.py:1473] [Warmup][Decode][278/325] batch_size:2 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:11 hpu_model_runner.py:1473] [Warmup][Decode][278/325] batch_size:2 num_blocks:1664 free_mem:22.86 GiB
INFO 01-31 03:33:13 hpu_model_runner.py:1473] [Warmup][Decode][279/325] batch_size:1 num_blocks:3116 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:13 hpu_model_runner.py:1473] [Warmup][Decode][279/325] batch_size:1 num_blocks:3116 free_mem:22.86 GiB
INFO 01-31 03:33:16 hpu_model_runner.py:1473] [Warmup][Decode][280/325] batch_size:1 num_blocks:3072 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:16 hpu_model_runner.py:1473] [Warmup][Decode][280/325] batch_size:1 num_blocks:3072 free_mem:22.86 GiB
INFO 01-31 03:33:20 hpu_model_runner.py:1473] [Warmup][Decode][281/325] batch_size:2 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:20 hpu_model_runner.py:1473] [Warmup][Decode][281/325] batch_size:2 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:33:22 hpu_model_runner.py:1473] [Warmup][Decode][282/325] batch_size:4 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:22 hpu_model_runner.py:1473] [Warmup][Decode][282/325] batch_size:4 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:33:24 hpu_model_runner.py:1473] [Warmup][Decode][283/325] batch_size:8 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:24 hpu_model_runner.py:1473] [Warmup][Decode][283/325] batch_size:8 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:33:25 hpu_model_runner.py:1473] [Warmup][Decode][284/325] batch_size:1 num_blocks:2944 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:25 hpu_model_runner.py:1473] [Warmup][Decode][284/325] batch_size:1 num_blocks:2944 free_mem:22.86 GiB
INFO 01-31 03:33:29 hpu_model_runner.py:1473] [Warmup][Decode][285/325] batch_size:1 num_blocks:2816 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:29 hpu_model_runner.py:1473] [Warmup][Decode][285/325] batch_size:1 num_blocks:2816 free_mem:22.86 GiB
INFO 01-31 03:33:32 hpu_model_runner.py:1473] [Warmup][Decode][286/325] batch_size:2 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:32 hpu_model_runner.py:1473] [Warmup][Decode][286/325] batch_size:2 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:34 hpu_model_runner.py:1473] [Warmup][Decode][287/325] batch_size:1 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:33:34 hpu_model_runner.py:1473] [Warmup][Decode][287/325] batch_size:1 num_blocks:2688 free_mem:22.86 GiB
INFO 01-31 03:33:37 hpu_model_runner.py:1473] [Warmup][Decode][288/325] batch_size:1 num_blocks:2560 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:37 hpu_model_runner.py:1473] [Warmup][Decode][288/325] batch_size:1 num_blocks:2560 free_mem:22.86 GiB
INFO 01-31 03:33:40 hpu_model_runner.py:1473] [Warmup][Decode][289/325] batch_size:2 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:40 hpu_model_runner.py:1473] [Warmup][Decode][289/325] batch_size:2 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:33:42 hpu_model_runner.py:1473] [Warmup][Decode][290/325] batch_size:4 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:42 hpu_model_runner.py:1473] [Warmup][Decode][290/325] batch_size:4 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:33:44 hpu_model_runner.py:1473] [Warmup][Decode][291/325] batch_size:1 num_blocks:2432 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:44 hpu_model_runner.py:1473] [Warmup][Decode][291/325] batch_size:1 num_blocks:2432 free_mem:22.86 GiB
INFO 01-31 03:33:47 hpu_model_runner.py:1473] [Warmup][Decode][292/325] batch_size:1 num_blocks:2304 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:47 hpu_model_runner.py:1473] [Warmup][Decode][292/325] batch_size:1 num_blocks:2304 free_mem:22.86 GiB
INFO 01-31 03:33:50 hpu_model_runner.py:1473] [Warmup][Decode][293/325] batch_size:2 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:50 hpu_model_runner.py:1473] [Warmup][Decode][293/325] batch_size:2 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:33:52 hpu_model_runner.py:1473] [Warmup][Decode][294/325] batch_size:1 num_blocks:2176 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:52 hpu_model_runner.py:1473] [Warmup][Decode][294/325] batch_size:1 num_blocks:2176 free_mem:22.86 GiB
INFO 01-31 03:33:55 hpu_model_runner.py:1473] [Warmup][Decode][295/325] batch_size:1 num_blocks:2048 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:55 hpu_model_runner.py:1473] [Warmup][Decode][295/325] batch_size:1 num_blocks:2048 free_mem:22.86 GiB
INFO 01-31 03:33:57 hpu_model_runner.py:1473] [Warmup][Decode][296/325] batch_size:2 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:57 hpu_model_runner.py:1473] [Warmup][Decode][296/325] batch_size:2 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:33:59 hpu_model_runner.py:1473] [Warmup][Decode][297/325] batch_size:4 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:33:59 hpu_model_runner.py:1473] [Warmup][Decode][297/325] batch_size:4 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:34:01 hpu_model_runner.py:1473] [Warmup][Decode][298/325] batch_size:8 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:01 hpu_model_runner.py:1473] [Warmup][Decode][298/325] batch_size:8 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:34:02 hpu_model_runner.py:1473] [Warmup][Decode][299/325] batch_size:16 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:02 hpu_model_runner.py:1473] [Warmup][Decode][299/325] batch_size:16 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:34:04 hpu_model_runner.py:1473] [Warmup][Decode][300/325] batch_size:1 num_blocks:1920 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:04 hpu_model_runner.py:1473] [Warmup][Decode][300/325] batch_size:1 num_blocks:1920 free_mem:22.86 GiB
INFO 01-31 03:34:06 hpu_model_runner.py:1473] [Warmup][Decode][301/325] batch_size:1 num_blocks:1792 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:06 hpu_model_runner.py:1473] [Warmup][Decode][301/325] batch_size:1 num_blocks:1792 free_mem:22.86 GiB
INFO 01-31 03:34:09 hpu_model_runner.py:1473] [Warmup][Decode][302/325] batch_size:2 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:09 hpu_model_runner.py:1473] [Warmup][Decode][302/325] batch_size:2 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:34:11 hpu_model_runner.py:1473] [Warmup][Decode][303/325] batch_size:1 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:11 hpu_model_runner.py:1473] [Warmup][Decode][303/325] batch_size:1 num_blocks:1664 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:13 hpu_model_runner.py:1473] [Warmup][Decode][304/325] batch_size:1 num_blocks:1536 free_mem:22.86 GiB
INFO 01-31 03:34:13 hpu_model_runner.py:1473] [Warmup][Decode][304/325] batch_size:1 num_blocks:1536 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:15 hpu_model_runner.py:1473] [Warmup][Decode][305/325] batch_size:2 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:34:15 hpu_model_runner.py:1473] [Warmup][Decode][305/325] batch_size:2 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:17 hpu_model_runner.py:1473] [Warmup][Decode][306/325] batch_size:4 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:34:17 hpu_model_runner.py:1473] [Warmup][Decode][306/325] batch_size:4 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:34:18 hpu_model_runner.py:1473] [Warmup][Decode][307/325] batch_size:1 num_blocks:1408 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:18 hpu_model_runner.py:1473] [Warmup][Decode][307/325] batch_size:1 num_blocks:1408 free_mem:22.86 GiB
INFO 01-31 03:34:21 hpu_model_runner.py:1473] [Warmup][Decode][308/325] batch_size:1 num_blocks:1280 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:21 hpu_model_runner.py:1473] [Warmup][Decode][308/325] batch_size:1 num_blocks:1280 free_mem:22.86 GiB
INFO 01-31 03:34:23 hpu_model_runner.py:1473] [Warmup][Decode][309/325] batch_size:2 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:23 hpu_model_runner.py:1473] [Warmup][Decode][309/325] batch_size:2 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:34:24 hpu_model_runner.py:1473] [Warmup][Decode][310/325] batch_size:1 num_blocks:1152 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:24 hpu_model_runner.py:1473] [Warmup][Decode][310/325] batch_size:1 num_blocks:1152 free_mem:22.86 GiB
INFO 01-31 03:34:26 hpu_model_runner.py:1473] [Warmup][Decode][311/325] batch_size:1 num_blocks:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:26 hpu_model_runner.py:1473] [Warmup][Decode][311/325] batch_size:1 num_blocks:1024 free_mem:22.86 GiB
INFO 01-31 03:34:28 hpu_model_runner.py:1473] [Warmup][Decode][312/325] batch_size:2 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:28 hpu_model_runner.py:1473] [Warmup][Decode][312/325] batch_size:2 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:34:30 hpu_model_runner.py:1473] [Warmup][Decode][313/325] batch_size:4 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:30 hpu_model_runner.py:1473] [Warmup][Decode][313/325] batch_size:4 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:34:31 hpu_model_runner.py:1473] [Warmup][Decode][314/325] batch_size:8 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:31 hpu_model_runner.py:1473] [Warmup][Decode][314/325] batch_size:8 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:34:33 hpu_model_runner.py:1473] [Warmup][Decode][315/325] batch_size:1 num_blocks:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:33 hpu_model_runner.py:1473] [Warmup][Decode][315/325] batch_size:1 num_blocks:896 free_mem:22.86 GiB
INFO 01-31 03:34:34 hpu_model_runner.py:1473] [Warmup][Decode][316/325] batch_size:1 num_blocks:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:35 hpu_model_runner.py:1473] [Warmup][Decode][316/325] batch_size:1 num_blocks:768 free_mem:22.86 GiB
INFO 01-31 03:34:36 hpu_model_runner.py:1473] [Warmup][Decode][317/325] batch_size:2 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:36 hpu_model_runner.py:1473] [Warmup][Decode][317/325] batch_size:2 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:34:38 hpu_model_runner.py:1473] [Warmup][Decode][318/325] batch_size:1 num_blocks:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:38 hpu_model_runner.py:1473] [Warmup][Decode][318/325] batch_size:1 num_blocks:640 free_mem:22.86 GiB
INFO 01-31 03:34:39 hpu_model_runner.py:1473] [Warmup][Decode][319/325] batch_size:1 num_blocks:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:39 hpu_model_runner.py:1473] [Warmup][Decode][319/325] batch_size:1 num_blocks:512 free_mem:22.86 GiB
INFO 01-31 03:34:41 hpu_model_runner.py:1473] [Warmup][Decode][320/325] batch_size:2 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:41 hpu_model_runner.py:1473] [Warmup][Decode][320/325] batch_size:2 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:34:42 hpu_model_runner.py:1473] [Warmup][Decode][321/325] batch_size:4 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:42 hpu_model_runner.py:1473] [Warmup][Decode][321/325] batch_size:4 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:34:44 hpu_model_runner.py:1473] [Warmup][Decode][322/325] batch_size:1 num_blocks:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:44 hpu_model_runner.py:1473] [Warmup][Decode][322/325] batch_size:1 num_blocks:384 free_mem:22.86 GiB
INFO 01-31 03:34:45 hpu_model_runner.py:1473] [Warmup][Decode][323/325] batch_size:1 num_blocks:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:45 hpu_model_runner.py:1473] [Warmup][Decode][323/325] batch_size:1 num_blocks:256 free_mem:22.86 GiB
INFO 01-31 03:34:46 hpu_model_runner.py:1473] [Warmup][Decode][324/325] batch_size:2 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:46 hpu_model_runner.py:1473] [Warmup][Decode][324/325] batch_size:2 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:48 hpu_model_runner.py:1473] [Warmup][Decode][325/325] batch_size:1 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:34:48 hpu_model_runner.py:1473] [Warmup][Decode][325/325] batch_size:1 num_blocks:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:49 hpu_model_runner.py:1621] Using 10.84 GiB/22.86 GiB of free device memory for HPUGraphs, 3.251 GiB for prompt and 7.585 GiB for decode (VLLM_GRAPH_PROMPT_RATIO=0.3)
INFO 01-31 03:34:49 hpu_model_runner.py:1621] Using 10.84 GiB/22.86 GiB of free device memory for HPUGraphs, 3.251 GiB for prompt and 7.585 GiB for decode (VLLM_GRAPH_PROMPT_RATIO=0.3)
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:49 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][1/31] batch_size:1 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:34:49 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][1/31] batch_size:1 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:34:50 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][2/31] batch_size:2 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:50 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][2/31] batch_size:2 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:34:51 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][3/31] batch_size:1 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:51 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][3/31] batch_size:1 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:34:52 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][4/31] batch_size:1 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:52 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][4/31] batch_size:1 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:34:52 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][5/31] batch_size:4 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:52 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][5/31] batch_size:4 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:34:53 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][6/31] batch_size:2 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:53 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][6/31] batch_size:2 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:34:54 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][7/31] batch_size:1 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:54 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][7/31] batch_size:1 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:55 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][8/31] batch_size:1 seq_len:640 free_mem:22.86 GiB
INFO 01-31 03:34:55 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][8/31] batch_size:1 seq_len:640 free_mem:22.86 GiB
INFO 01-31 03:34:56 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][9/31] batch_size:2 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:56 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][9/31] batch_size:2 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:34:56 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][10/31] batch_size:1 seq_len:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:56 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][10/31] batch_size:1 seq_len:768 free_mem:22.86 GiB
INFO 01-31 03:34:57 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][11/31] batch_size:1 seq_len:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:57 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][11/31] batch_size:1 seq_len:896 free_mem:22.86 GiB
INFO 01-31 03:34:58 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][12/31] batch_size:8 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:58 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][12/31] batch_size:8 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:34:59 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][13/31] batch_size:4 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:34:59 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][13/31] batch_size:4 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:35:00 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][14/31] batch_size:2 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:00 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][14/31] batch_size:2 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:00 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][15/31] batch_size:1 seq_len:1024 free_mem:22.86 GiB
INFO 01-31 03:35:00 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][15/31] batch_size:1 seq_len:1024 free_mem:22.86 GiB
INFO 01-31 03:35:01 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][16/31] batch_size:2 seq_len:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:01 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][16/31] batch_size:2 seq_len:640 free_mem:22.86 GiB
INFO 01-31 03:35:02 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][17/31] batch_size:4 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:02 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][17/31] batch_size:4 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:35:03 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][18/31] batch_size:2 seq_len:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:03 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][18/31] batch_size:2 seq_len:768 free_mem:22.86 GiB
INFO 01-31 03:35:04 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][19/31] batch_size:2 seq_len:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:04 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][19/31] batch_size:2 seq_len:896 free_mem:22.86 GiB
INFO 01-31 03:35:04 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][20/31] batch_size:16 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:04 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][20/31] batch_size:16 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:35:05 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][21/31] batch_size:8 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:05 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][21/31] batch_size:8 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:35:06 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][22/31] batch_size:4 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:06 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][22/31] batch_size:4 seq_len:512 free_mem:22.86 GiB
INFO 01-31 03:35:07 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][23/31] batch_size:2 seq_len:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:07 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][23/31] batch_size:2 seq_len:1024 free_mem:22.86 GiB
INFO 01-31 03:35:08 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][24/31] batch_size:4 seq_len:640 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:08 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][24/31] batch_size:4 seq_len:640 free_mem:22.86 GiB
INFO 01-31 03:35:09 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][25/31] batch_size:8 seq_len:384 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:09 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][25/31] batch_size:8 seq_len:384 free_mem:22.86 GiB
INFO 01-31 03:35:10 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][26/31] batch_size:4 seq_len:768 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:10 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][26/31] batch_size:4 seq_len:768 free_mem:22.86 GiB
INFO 01-31 03:35:11 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][27/31] batch_size:4 seq_len:896 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:11 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][27/31] batch_size:4 seq_len:896 free_mem:22.86 GiB
INFO 01-31 03:35:11 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][28/31] batch_size:32 seq_len:128 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:11 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][28/31] batch_size:32 seq_len:128 free_mem:22.86 GiB
INFO 01-31 03:35:13 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][29/31] batch_size:16 seq_len:256 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:13 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][29/31] batch_size:16 seq_len:256 free_mem:22.86 GiB
INFO 01-31 03:35:14 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][30/31] batch_size:8 seq_len:512 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:14 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][30/31] batch_size:8 seq_len:512 free_mem:22.86 GiB
INFO 01-31 03:35:15 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][31/31] batch_size:4 seq_len:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:15 hpu_model_runner.py:1473] [Warmup][Graph/Prompt][31/31] batch_size:4 seq_len:1024 free_mem:22.86 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][1/325] batch_size:256 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:35:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][1/325] batch_size:256 num_blocks:128 free_mem:22.86 GiB
INFO 01-31 03:35:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][2/325] batch_size:256 num_blocks:256 free_mem:22.85 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][2/325] batch_size:256 num_blocks:256 free_mem:22.85 GiB
INFO 01-31 03:35:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][3/325] batch_size:256 num_blocks:384 free_mem:22.85 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][3/325] batch_size:256 num_blocks:384 free_mem:22.85 GiB
INFO 01-31 03:35:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][4/325] batch_size:256 num_blocks:512 free_mem:22.85 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][4/325] batch_size:256 num_blocks:512 free_mem:22.85 GiB
INFO 01-31 03:35:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][5/325] batch_size:256 num_blocks:640 free_mem:22.85 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][5/325] batch_size:256 num_blocks:640 free_mem:22.85 GiB
INFO 01-31 03:35:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][6/325] batch_size:256 num_blocks:768 free_mem:22.85 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][6/325] batch_size:256 num_blocks:768 free_mem:22.85 GiB
INFO 01-31 03:35:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][7/325] batch_size:256 num_blocks:896 free_mem:22.84 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][7/325] batch_size:256 num_blocks:896 free_mem:22.84 GiB
INFO 01-31 03:35:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][8/325] batch_size:256 num_blocks:1024 free_mem:22.84 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][8/325] batch_size:256 num_blocks:1024 free_mem:22.84 GiB
INFO 01-31 03:35:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][9/325] batch_size:256 num_blocks:1152 free_mem:22.84 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][9/325] batch_size:256 num_blocks:1152 free_mem:22.84 GiB
INFO 01-31 03:35:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][10/325] batch_size:256 num_blocks:1280 free_mem:22.84 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][10/325] batch_size:256 num_blocks:1280 free_mem:22.84 GiB
INFO 01-31 03:35:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][11/325] batch_size:256 num_blocks:1408 free_mem:22.83 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][11/325] batch_size:256 num_blocks:1408 free_mem:22.83 GiB
INFO 01-31 03:35:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][12/325] batch_size:256 num_blocks:1536 free_mem:22.83 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][12/325] batch_size:256 num_blocks:1536 free_mem:22.83 GiB
INFO 01-31 03:35:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][13/325] batch_size:256 num_blocks:1664 free_mem:22.83 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][13/325] batch_size:256 num_blocks:1664 free_mem:22.83 GiB
INFO 01-31 03:35:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][14/325] batch_size:256 num_blocks:1792 free_mem:22.83 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][14/325] batch_size:256 num_blocks:1792 free_mem:22.83 GiB
INFO 01-31 03:35:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][15/325] batch_size:256 num_blocks:1920 free_mem:22.82 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][15/325] batch_size:256 num_blocks:1920 free_mem:22.82 GiB
INFO 01-31 03:35:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][16/325] batch_size:256 num_blocks:2048 free_mem:22.82 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][16/325] batch_size:256 num_blocks:2048 free_mem:22.82 GiB
INFO 01-31 03:35:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][17/325] batch_size:256 num_blocks:2176 free_mem:22.82 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][17/325] batch_size:256 num_blocks:2176 free_mem:22.82 GiB
INFO 01-31 03:35:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][18/325] batch_size:256 num_blocks:2304 free_mem:22.81 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][18/325] batch_size:256 num_blocks:2304 free_mem:22.81 GiB
INFO 01-31 03:35:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][19/325] batch_size:256 num_blocks:2432 free_mem:22.81 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][19/325] batch_size:256 num_blocks:2432 free_mem:22.81 GiB
INFO 01-31 03:35:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][20/325] batch_size:256 num_blocks:2560 free_mem:22.81 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][20/325] batch_size:256 num_blocks:2560 free_mem:22.81 GiB
INFO 01-31 03:35:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][21/325] batch_size:256 num_blocks:2688 free_mem:22.81 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][21/325] batch_size:256 num_blocks:2688 free_mem:22.81 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][22/325] batch_size:256 num_blocks:2816 free_mem:22.8 GiB
INFO 01-31 03:35:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][22/325] batch_size:256 num_blocks:2816 free_mem:22.8 GiB
INFO 01-31 03:35:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][23/325] batch_size:256 num_blocks:2944 free_mem:22.8 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][23/325] batch_size:256 num_blocks:2944 free_mem:22.8 GiB
INFO 01-31 03:35:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][24/325] batch_size:256 num_blocks:3072 free_mem:22.8 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][24/325] batch_size:256 num_blocks:3072 free_mem:22.8 GiB
INFO 01-31 03:35:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][25/325] batch_size:256 num_blocks:3116 free_mem:22.79 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][25/325] batch_size:256 num_blocks:3116 free_mem:22.79 GiB
INFO 01-31 03:35:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][26/325] batch_size:224 num_blocks:128 free_mem:22.79 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][26/325] batch_size:224 num_blocks:128 free_mem:22.79 GiB
INFO 01-31 03:35:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][27/325] batch_size:224 num_blocks:256 free_mem:22.79 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][27/325] batch_size:224 num_blocks:256 free_mem:22.79 GiB
INFO 01-31 03:35:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][28/325] batch_size:224 num_blocks:384 free_mem:22.79 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][28/325] batch_size:224 num_blocks:384 free_mem:22.79 GiB
INFO 01-31 03:35:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][29/325] batch_size:224 num_blocks:512 free_mem:22.79 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][29/325] batch_size:224 num_blocks:512 free_mem:22.79 GiB
INFO 01-31 03:35:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][30/325] batch_size:224 num_blocks:640 free_mem:22.78 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][30/325] batch_size:224 num_blocks:640 free_mem:22.78 GiB
INFO 01-31 03:35:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][31/325] batch_size:224 num_blocks:768 free_mem:22.78 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][31/325] batch_size:224 num_blocks:768 free_mem:22.78 GiB
INFO 01-31 03:35:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][32/325] batch_size:224 num_blocks:896 free_mem:22.78 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][32/325] batch_size:224 num_blocks:896 free_mem:22.78 GiB
INFO 01-31 03:35:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][33/325] batch_size:224 num_blocks:1024 free_mem:22.78 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][33/325] batch_size:224 num_blocks:1024 free_mem:22.78 GiB
INFO 01-31 03:35:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][34/325] batch_size:224 num_blocks:1152 free_mem:22.78 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][34/325] batch_size:224 num_blocks:1152 free_mem:22.78 GiB
INFO 01-31 03:35:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][35/325] batch_size:224 num_blocks:1280 free_mem:22.77 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][35/325] batch_size:224 num_blocks:1280 free_mem:22.77 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][36/325] batch_size:224 num_blocks:1408 free_mem:22.77 GiB
INFO 01-31 03:35:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][36/325] batch_size:224 num_blocks:1408 free_mem:22.77 GiB
INFO 01-31 03:35:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][37/325] batch_size:224 num_blocks:1536 free_mem:22.77 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][37/325] batch_size:224 num_blocks:1536 free_mem:22.77 GiB
INFO 01-31 03:35:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][38/325] batch_size:224 num_blocks:1664 free_mem:22.77 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][38/325] batch_size:224 num_blocks:1664 free_mem:22.77 GiB
INFO 01-31 03:35:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][39/325] batch_size:224 num_blocks:1792 free_mem:22.76 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][39/325] batch_size:224 num_blocks:1792 free_mem:22.76 GiB
INFO 01-31 03:35:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][40/325] batch_size:224 num_blocks:1920 free_mem:22.76 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][40/325] batch_size:224 num_blocks:1920 free_mem:22.76 GiB
INFO 01-31 03:35:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][41/325] batch_size:224 num_blocks:2048 free_mem:22.76 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][41/325] batch_size:224 num_blocks:2048 free_mem:22.76 GiB
INFO 01-31 03:35:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][42/325] batch_size:224 num_blocks:2176 free_mem:22.76 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][42/325] batch_size:224 num_blocks:2176 free_mem:22.76 GiB
INFO 01-31 03:35:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][43/325] batch_size:224 num_blocks:2304 free_mem:22.75 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][43/325] batch_size:224 num_blocks:2304 free_mem:22.75 GiB
INFO 01-31 03:35:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][44/325] batch_size:224 num_blocks:2432 free_mem:22.75 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][44/325] batch_size:224 num_blocks:2432 free_mem:22.75 GiB
INFO 01-31 03:35:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][45/325] batch_size:224 num_blocks:2560 free_mem:22.75 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][45/325] batch_size:224 num_blocks:2560 free_mem:22.75 GiB
INFO 01-31 03:35:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][46/325] batch_size:224 num_blocks:2688 free_mem:22.75 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][46/325] batch_size:224 num_blocks:2688 free_mem:22.75 GiB
INFO 01-31 03:35:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][47/325] batch_size:224 num_blocks:2816 free_mem:22.74 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][47/325] batch_size:224 num_blocks:2816 free_mem:22.74 GiB
INFO 01-31 03:35:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][48/325] batch_size:224 num_blocks:2944 free_mem:22.74 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][48/325] batch_size:224 num_blocks:2944 free_mem:22.74 GiB
INFO 01-31 03:35:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][49/325] batch_size:224 num_blocks:3072 free_mem:22.74 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][49/325] batch_size:224 num_blocks:3072 free_mem:22.74 GiB
INFO 01-31 03:35:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][50/325] batch_size:224 num_blocks:3116 free_mem:22.74 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][50/325] batch_size:224 num_blocks:3116 free_mem:22.74 GiB
INFO 01-31 03:35:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][51/325] batch_size:192 num_blocks:128 free_mem:22.73 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][51/325] batch_size:192 num_blocks:128 free_mem:22.73 GiB
INFO 01-31 03:35:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][52/325] batch_size:192 num_blocks:256 free_mem:22.73 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][52/325] batch_size:192 num_blocks:256 free_mem:22.73 GiB
INFO 01-31 03:35:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][53/325] batch_size:192 num_blocks:384 free_mem:22.73 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][53/325] batch_size:192 num_blocks:384 free_mem:22.73 GiB
INFO 01-31 03:35:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][54/325] batch_size:192 num_blocks:512 free_mem:22.73 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][54/325] batch_size:192 num_blocks:512 free_mem:22.73 GiB
INFO 01-31 03:35:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][55/325] batch_size:192 num_blocks:640 free_mem:22.73 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:35:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][55/325] batch_size:192 num_blocks:640 free_mem:22.73 GiB
INFO 01-31 03:36:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][56/325] batch_size:192 num_blocks:768 free_mem:22.72 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][56/325] batch_size:192 num_blocks:768 free_mem:22.72 GiB
INFO 01-31 03:36:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][57/325] batch_size:192 num_blocks:896 free_mem:22.72 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][57/325] batch_size:192 num_blocks:896 free_mem:22.72 GiB
INFO 01-31 03:36:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][58/325] batch_size:192 num_blocks:1024 free_mem:22.72 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][58/325] batch_size:192 num_blocks:1024 free_mem:22.72 GiB
INFO 01-31 03:36:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][59/325] batch_size:192 num_blocks:1152 free_mem:22.72 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][59/325] batch_size:192 num_blocks:1152 free_mem:22.72 GiB
INFO 01-31 03:36:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][60/325] batch_size:192 num_blocks:1280 free_mem:22.72 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][60/325] batch_size:192 num_blocks:1280 free_mem:22.72 GiB
INFO 01-31 03:36:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][61/325] batch_size:192 num_blocks:1408 free_mem:22.72 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][61/325] batch_size:192 num_blocks:1408 free_mem:22.72 GiB
INFO 01-31 03:36:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][62/325] batch_size:192 num_blocks:1536 free_mem:22.71 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][62/325] batch_size:192 num_blocks:1536 free_mem:22.71 GiB
INFO 01-31 03:36:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][63/325] batch_size:192 num_blocks:1664 free_mem:22.71 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][63/325] batch_size:192 num_blocks:1664 free_mem:22.71 GiB
INFO 01-31 03:36:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][64/325] batch_size:192 num_blocks:1792 free_mem:22.71 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][64/325] batch_size:192 num_blocks:1792 free_mem:22.71 GiB
INFO 01-31 03:36:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][65/325] batch_size:192 num_blocks:1920 free_mem:22.71 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][65/325] batch_size:192 num_blocks:1920 free_mem:22.71 GiB
INFO 01-31 03:36:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][66/325] batch_size:192 num_blocks:2048 free_mem:22.7 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][66/325] batch_size:192 num_blocks:2048 free_mem:22.7 GiB
INFO 01-31 03:36:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][67/325] batch_size:192 num_blocks:2176 free_mem:22.7 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][67/325] batch_size:192 num_blocks:2176 free_mem:22.7 GiB
INFO 01-31 03:36:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][68/325] batch_size:192 num_blocks:2304 free_mem:22.7 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][68/325] batch_size:192 num_blocks:2304 free_mem:22.7 GiB
INFO 01-31 03:36:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][69/325] batch_size:192 num_blocks:2432 free_mem:22.7 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][69/325] batch_size:192 num_blocks:2432 free_mem:22.7 GiB
INFO 01-31 03:36:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][70/325] batch_size:192 num_blocks:2560 free_mem:22.7 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][70/325] batch_size:192 num_blocks:2560 free_mem:22.7 GiB
INFO 01-31 03:36:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][71/325] batch_size:192 num_blocks:2688 free_mem:22.69 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][71/325] batch_size:192 num_blocks:2688 free_mem:22.69 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][72/325] batch_size:192 num_blocks:2816 free_mem:22.69 GiB
INFO 01-31 03:36:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][72/325] batch_size:192 num_blocks:2816 free_mem:22.69 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][73/325] batch_size:192 num_blocks:2944 free_mem:22.69 GiB
INFO 01-31 03:36:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][73/325] batch_size:192 num_blocks:2944 free_mem:22.69 GiB
INFO 01-31 03:36:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][74/325] batch_size:192 num_blocks:3072 free_mem:22.69 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][74/325] batch_size:192 num_blocks:3072 free_mem:22.69 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][75/325] batch_size:192 num_blocks:3116 free_mem:22.68 GiB
INFO 01-31 03:36:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][75/325] batch_size:192 num_blocks:3116 free_mem:22.68 GiB
INFO 01-31 03:36:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][76/325] batch_size:160 num_blocks:128 free_mem:22.68 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][76/325] batch_size:160 num_blocks:128 free_mem:22.68 GiB
INFO 01-31 03:36:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][77/325] batch_size:160 num_blocks:256 free_mem:22.68 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][77/325] batch_size:160 num_blocks:256 free_mem:22.68 GiB
INFO 01-31 03:36:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][78/325] batch_size:160 num_blocks:384 free_mem:22.68 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][78/325] batch_size:160 num_blocks:384 free_mem:22.68 GiB
INFO 01-31 03:36:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][79/325] batch_size:160 num_blocks:512 free_mem:22.68 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][79/325] batch_size:160 num_blocks:512 free_mem:22.68 GiB
INFO 01-31 03:36:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][80/325] batch_size:160 num_blocks:640 free_mem:22.68 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][80/325] batch_size:160 num_blocks:640 free_mem:22.68 GiB
INFO 01-31 03:36:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][81/325] batch_size:160 num_blocks:768 free_mem:22.67 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][81/325] batch_size:160 num_blocks:768 free_mem:22.67 GiB
INFO 01-31 03:36:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][82/325] batch_size:160 num_blocks:896 free_mem:22.67 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][82/325] batch_size:160 num_blocks:896 free_mem:22.67 GiB
INFO 01-31 03:36:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][83/325] batch_size:160 num_blocks:1024 free_mem:22.67 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][83/325] batch_size:160 num_blocks:1024 free_mem:22.67 GiB
INFO 01-31 03:36:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][84/325] batch_size:160 num_blocks:1152 free_mem:22.67 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][84/325] batch_size:160 num_blocks:1152 free_mem:22.67 GiB
INFO 01-31 03:36:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][85/325] batch_size:160 num_blocks:1280 free_mem:22.67 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][85/325] batch_size:160 num_blocks:1280 free_mem:22.67 GiB
INFO 01-31 03:36:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][86/325] batch_size:160 num_blocks:1408 free_mem:22.67 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][86/325] batch_size:160 num_blocks:1408 free_mem:22.67 GiB
INFO 01-31 03:36:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][87/325] batch_size:160 num_blocks:1536 free_mem:22.66 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][87/325] batch_size:160 num_blocks:1536 free_mem:22.66 GiB
INFO 01-31 03:36:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][88/325] batch_size:160 num_blocks:1664 free_mem:22.66 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][88/325] batch_size:160 num_blocks:1664 free_mem:22.66 GiB
INFO 01-31 03:36:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][89/325] batch_size:160 num_blocks:1792 free_mem:22.66 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][89/325] batch_size:160 num_blocks:1792 free_mem:22.66 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][90/325] batch_size:160 num_blocks:1920 free_mem:22.66 GiB
INFO 01-31 03:36:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][90/325] batch_size:160 num_blocks:1920 free_mem:22.66 GiB
INFO 01-31 03:36:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][91/325] batch_size:160 num_blocks:2048 free_mem:22.66 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][91/325] batch_size:160 num_blocks:2048 free_mem:22.66 GiB
INFO 01-31 03:36:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][92/325] batch_size:160 num_blocks:2176 free_mem:22.66 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][92/325] batch_size:160 num_blocks:2176 free_mem:22.66 GiB
INFO 01-31 03:36:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][93/325] batch_size:160 num_blocks:2304 free_mem:22.65 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][93/325] batch_size:160 num_blocks:2304 free_mem:22.65 GiB
INFO 01-31 03:36:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][94/325] batch_size:160 num_blocks:2432 free_mem:22.65 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][94/325] batch_size:160 num_blocks:2432 free_mem:22.65 GiB
INFO 01-31 03:36:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][95/325] batch_size:160 num_blocks:2560 free_mem:22.65 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][95/325] batch_size:160 num_blocks:2560 free_mem:22.65 GiB
INFO 01-31 03:36:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][96/325] batch_size:160 num_blocks:2688 free_mem:22.65 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][96/325] batch_size:160 num_blocks:2688 free_mem:22.65 GiB
INFO 01-31 03:36:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][97/325] batch_size:160 num_blocks:2816 free_mem:22.65 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][97/325] batch_size:160 num_blocks:2816 free_mem:22.65 GiB
INFO 01-31 03:36:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][98/325] batch_size:160 num_blocks:2944 free_mem:22.64 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][98/325] batch_size:160 num_blocks:2944 free_mem:22.64 GiB
INFO 01-31 03:36:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][99/325] batch_size:160 num_blocks:3072 free_mem:22.64 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][99/325] batch_size:160 num_blocks:3072 free_mem:22.64 GiB
INFO 01-31 03:36:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][100/325] batch_size:160 num_blocks:3116 free_mem:22.64 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][100/325] batch_size:160 num_blocks:3116 free_mem:22.64 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][101/325] batch_size:128 num_blocks:128 free_mem:22.64 GiB
INFO 01-31 03:36:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][101/325] batch_size:128 num_blocks:128 free_mem:22.64 GiB
INFO 01-31 03:36:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][102/325] batch_size:128 num_blocks:256 free_mem:22.64 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][102/325] batch_size:128 num_blocks:256 free_mem:22.64 GiB
INFO 01-31 03:36:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][103/325] batch_size:128 num_blocks:384 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][103/325] batch_size:128 num_blocks:384 free_mem:22.63 GiB
INFO 01-31 03:36:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][104/325] batch_size:128 num_blocks:512 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][104/325] batch_size:128 num_blocks:512 free_mem:22.63 GiB
INFO 01-31 03:36:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][105/325] batch_size:128 num_blocks:640 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][105/325] batch_size:128 num_blocks:640 free_mem:22.63 GiB
INFO 01-31 03:36:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][106/325] batch_size:128 num_blocks:768 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][106/325] batch_size:128 num_blocks:768 free_mem:22.63 GiB
INFO 01-31 03:36:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][107/325] batch_size:128 num_blocks:896 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][107/325] batch_size:128 num_blocks:896 free_mem:22.63 GiB
INFO 01-31 03:36:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][108/325] batch_size:128 num_blocks:1024 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][108/325] batch_size:128 num_blocks:1024 free_mem:22.63 GiB
INFO 01-31 03:36:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][109/325] batch_size:128 num_blocks:1152 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][109/325] batch_size:128 num_blocks:1152 free_mem:22.63 GiB
INFO 01-31 03:36:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][110/325] batch_size:128 num_blocks:1280 free_mem:22.63 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][110/325] batch_size:128 num_blocks:1280 free_mem:22.63 GiB
INFO 01-31 03:36:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][111/325] batch_size:128 num_blocks:1408 free_mem:22.62 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][111/325] batch_size:128 num_blocks:1408 free_mem:22.62 GiB
INFO 01-31 03:36:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][112/325] batch_size:128 num_blocks:1536 free_mem:22.62 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][112/325] batch_size:128 num_blocks:1536 free_mem:22.62 GiB
INFO 01-31 03:36:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][113/325] batch_size:128 num_blocks:1664 free_mem:22.62 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][113/325] batch_size:128 num_blocks:1664 free_mem:22.62 GiB
INFO 01-31 03:36:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][114/325] batch_size:128 num_blocks:1792 free_mem:22.62 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][114/325] batch_size:128 num_blocks:1792 free_mem:22.62 GiB
INFO 01-31 03:36:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][115/325] batch_size:128 num_blocks:1920 free_mem:22.62 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][115/325] batch_size:128 num_blocks:1920 free_mem:22.62 GiB
INFO 01-31 03:36:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][116/325] batch_size:128 num_blocks:2048 free_mem:22.62 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][116/325] batch_size:128 num_blocks:2048 free_mem:22.62 GiB
INFO 01-31 03:36:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][117/325] batch_size:128 num_blocks:2176 free_mem:22.62 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][117/325] batch_size:128 num_blocks:2176 free_mem:22.62 GiB
INFO 01-31 03:36:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][118/325] batch_size:128 num_blocks:2304 free_mem:22.61 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][118/325] batch_size:128 num_blocks:2304 free_mem:22.61 GiB
INFO 01-31 03:36:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][119/325] batch_size:128 num_blocks:2432 free_mem:22.61 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][119/325] batch_size:128 num_blocks:2432 free_mem:22.61 GiB
INFO 01-31 03:36:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][120/325] batch_size:128 num_blocks:2560 free_mem:22.61 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][120/325] batch_size:128 num_blocks:2560 free_mem:22.61 GiB
INFO 01-31 03:36:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][121/325] batch_size:128 num_blocks:2688 free_mem:22.61 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][121/325] batch_size:128 num_blocks:2688 free_mem:22.61 GiB
INFO 01-31 03:36:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][122/325] batch_size:128 num_blocks:2816 free_mem:22.61 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][122/325] batch_size:128 num_blocks:2816 free_mem:22.61 GiB
INFO 01-31 03:36:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][123/325] batch_size:128 num_blocks:2944 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][123/325] batch_size:128 num_blocks:2944 free_mem:22.6 GiB
INFO 01-31 03:36:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][124/325] batch_size:128 num_blocks:3072 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][124/325] batch_size:128 num_blocks:3072 free_mem:22.6 GiB
INFO 01-31 03:36:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][125/325] batch_size:128 num_blocks:3116 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][125/325] batch_size:128 num_blocks:3116 free_mem:22.6 GiB
INFO 01-31 03:36:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][126/325] batch_size:96 num_blocks:128 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][126/325] batch_size:96 num_blocks:128 free_mem:22.6 GiB
INFO 01-31 03:36:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][127/325] batch_size:96 num_blocks:256 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][127/325] batch_size:96 num_blocks:256 free_mem:22.6 GiB
INFO 01-31 03:36:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][128/325] batch_size:96 num_blocks:384 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][128/325] batch_size:96 num_blocks:384 free_mem:22.6 GiB
INFO 01-31 03:36:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][129/325] batch_size:96 num_blocks:512 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][129/325] batch_size:96 num_blocks:512 free_mem:22.6 GiB
INFO 01-31 03:36:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][130/325] batch_size:96 num_blocks:640 free_mem:22.6 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][130/325] batch_size:96 num_blocks:640 free_mem:22.6 GiB
INFO 01-31 03:36:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][131/325] batch_size:96 num_blocks:768 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:36:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][131/325] batch_size:96 num_blocks:768 free_mem:22.59 GiB
INFO 01-31 03:37:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][132/325] batch_size:96 num_blocks:896 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][132/325] batch_size:96 num_blocks:896 free_mem:22.59 GiB
INFO 01-31 03:37:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][133/325] batch_size:96 num_blocks:1024 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][133/325] batch_size:96 num_blocks:1024 free_mem:22.59 GiB
INFO 01-31 03:37:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][134/325] batch_size:96 num_blocks:1152 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][134/325] batch_size:96 num_blocks:1152 free_mem:22.59 GiB
INFO 01-31 03:37:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][135/325] batch_size:96 num_blocks:1280 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][135/325] batch_size:96 num_blocks:1280 free_mem:22.59 GiB
INFO 01-31 03:37:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][136/325] batch_size:96 num_blocks:1408 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][136/325] batch_size:96 num_blocks:1408 free_mem:22.59 GiB
INFO 01-31 03:37:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][137/325] batch_size:96 num_blocks:1536 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][137/325] batch_size:96 num_blocks:1536 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][138/325] batch_size:96 num_blocks:1664 free_mem:22.59 GiB
INFO 01-31 03:37:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][138/325] batch_size:96 num_blocks:1664 free_mem:22.59 GiB
INFO 01-31 03:37:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][139/325] batch_size:96 num_blocks:1792 free_mem:22.59 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][139/325] batch_size:96 num_blocks:1792 free_mem:22.59 GiB
INFO 01-31 03:37:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][140/325] batch_size:96 num_blocks:1920 free_mem:22.58 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][140/325] batch_size:96 num_blocks:1920 free_mem:22.58 GiB
INFO 01-31 03:37:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][141/325] batch_size:96 num_blocks:2048 free_mem:22.58 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][141/325] batch_size:96 num_blocks:2048 free_mem:22.58 GiB
INFO 01-31 03:37:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][142/325] batch_size:96 num_blocks:2176 free_mem:22.58 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][142/325] batch_size:96 num_blocks:2176 free_mem:22.58 GiB
INFO 01-31 03:37:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][143/325] batch_size:96 num_blocks:2304 free_mem:22.58 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][143/325] batch_size:96 num_blocks:2304 free_mem:22.58 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][144/325] batch_size:96 num_blocks:2432 free_mem:22.58 GiB
INFO 01-31 03:37:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][144/325] batch_size:96 num_blocks:2432 free_mem:22.58 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][145/325] batch_size:96 num_blocks:2560 free_mem:22.58 GiB
INFO 01-31 03:37:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][145/325] batch_size:96 num_blocks:2560 free_mem:22.58 GiB
INFO 01-31 03:37:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][146/325] batch_size:96 num_blocks:2688 free_mem:22.58 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][146/325] batch_size:96 num_blocks:2688 free_mem:22.58 GiB
INFO 01-31 03:37:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][147/325] batch_size:96 num_blocks:2816 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][147/325] batch_size:96 num_blocks:2816 free_mem:22.57 GiB
INFO 01-31 03:37:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][148/325] batch_size:96 num_blocks:2944 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][148/325] batch_size:96 num_blocks:2944 free_mem:22.57 GiB
INFO 01-31 03:37:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][149/325] batch_size:96 num_blocks:3072 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][149/325] batch_size:96 num_blocks:3072 free_mem:22.57 GiB
INFO 01-31 03:37:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][150/325] batch_size:96 num_blocks:3116 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][150/325] batch_size:96 num_blocks:3116 free_mem:22.57 GiB
INFO 01-31 03:37:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][151/325] batch_size:64 num_blocks:128 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][151/325] batch_size:64 num_blocks:128 free_mem:22.57 GiB
INFO 01-31 03:37:16 hpu_model_runner.py:1473] [Warmup][Graph/Decode][152/325] batch_size:64 num_blocks:256 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:16 hpu_model_runner.py:1473] [Warmup][Graph/Decode][152/325] batch_size:64 num_blocks:256 free_mem:22.57 GiB
INFO 01-31 03:37:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][153/325] batch_size:64 num_blocks:384 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][153/325] batch_size:64 num_blocks:384 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][154/325] batch_size:64 num_blocks:512 free_mem:22.57 GiB
INFO 01-31 03:37:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][154/325] batch_size:64 num_blocks:512 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][155/325] batch_size:64 num_blocks:640 free_mem:22.57 GiB
INFO 01-31 03:37:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][155/325] batch_size:64 num_blocks:640 free_mem:22.57 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][156/325] batch_size:64 num_blocks:768 free_mem:22.57 GiB
INFO 01-31 03:37:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][156/325] batch_size:64 num_blocks:768 free_mem:22.57 GiB
INFO 01-31 03:37:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][157/325] batch_size:64 num_blocks:896 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][157/325] batch_size:64 num_blocks:896 free_mem:22.56 GiB
INFO 01-31 03:37:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][158/325] batch_size:64 num_blocks:1024 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][158/325] batch_size:64 num_blocks:1024 free_mem:22.56 GiB
INFO 01-31 03:37:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][159/325] batch_size:64 num_blocks:1152 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][159/325] batch_size:64 num_blocks:1152 free_mem:22.56 GiB
INFO 01-31 03:37:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][160/325] batch_size:64 num_blocks:1280 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][160/325] batch_size:64 num_blocks:1280 free_mem:22.56 GiB
INFO 01-31 03:37:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][161/325] batch_size:64 num_blocks:1408 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][161/325] batch_size:64 num_blocks:1408 free_mem:22.56 GiB
INFO 01-31 03:37:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][162/325] batch_size:64 num_blocks:1536 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][162/325] batch_size:64 num_blocks:1536 free_mem:22.56 GiB
INFO 01-31 03:37:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][163/325] batch_size:64 num_blocks:1664 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][163/325] batch_size:64 num_blocks:1664 free_mem:22.56 GiB
INFO 01-31 03:37:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][164/325] batch_size:64 num_blocks:1792 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][164/325] batch_size:64 num_blocks:1792 free_mem:22.56 GiB
INFO 01-31 03:37:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][165/325] batch_size:64 num_blocks:1920 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][165/325] batch_size:64 num_blocks:1920 free_mem:22.56 GiB
INFO 01-31 03:37:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][166/325] batch_size:64 num_blocks:2048 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][166/325] batch_size:64 num_blocks:2048 free_mem:22.56 GiB
INFO 01-31 03:37:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][167/325] batch_size:64 num_blocks:2176 free_mem:22.56 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][167/325] batch_size:64 num_blocks:2176 free_mem:22.56 GiB
INFO 01-31 03:37:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][168/325] batch_size:64 num_blocks:2304 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][168/325] batch_size:64 num_blocks:2304 free_mem:22.55 GiB
INFO 01-31 03:37:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][169/325] batch_size:64 num_blocks:2432 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][169/325] batch_size:64 num_blocks:2432 free_mem:22.55 GiB
INFO 01-31 03:37:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][170/325] batch_size:64 num_blocks:2560 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][170/325] batch_size:64 num_blocks:2560 free_mem:22.55 GiB
INFO 01-31 03:37:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][171/325] batch_size:64 num_blocks:2688 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][171/325] batch_size:64 num_blocks:2688 free_mem:22.55 GiB
INFO 01-31 03:37:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][172/325] batch_size:64 num_blocks:2816 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][172/325] batch_size:64 num_blocks:2816 free_mem:22.55 GiB
INFO 01-31 03:37:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][173/325] batch_size:64 num_blocks:2944 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][173/325] batch_size:64 num_blocks:2944 free_mem:22.55 GiB
INFO 01-31 03:37:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][174/325] batch_size:64 num_blocks:3072 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][174/325] batch_size:64 num_blocks:3072 free_mem:22.55 GiB
INFO 01-31 03:37:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][175/325] batch_size:64 num_blocks:3116 free_mem:22.55 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][175/325] batch_size:64 num_blocks:3116 free_mem:22.55 GiB
INFO 01-31 03:37:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][176/325] batch_size:32 num_blocks:128 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][176/325] batch_size:32 num_blocks:128 free_mem:22.54 GiB
INFO 01-31 03:37:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][177/325] batch_size:32 num_blocks:256 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][177/325] batch_size:32 num_blocks:256 free_mem:22.54 GiB
INFO 01-31 03:37:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][178/325] batch_size:32 num_blocks:384 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][178/325] batch_size:32 num_blocks:384 free_mem:22.54 GiB
INFO 01-31 03:37:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][179/325] batch_size:32 num_blocks:512 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][179/325] batch_size:32 num_blocks:512 free_mem:22.54 GiB
INFO 01-31 03:37:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][180/325] batch_size:32 num_blocks:640 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][180/325] batch_size:32 num_blocks:640 free_mem:22.54 GiB
INFO 01-31 03:37:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][181/325] batch_size:32 num_blocks:768 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][181/325] batch_size:32 num_blocks:768 free_mem:22.54 GiB
INFO 01-31 03:37:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][182/325] batch_size:32 num_blocks:896 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][182/325] batch_size:32 num_blocks:896 free_mem:22.54 GiB
INFO 01-31 03:37:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][183/325] batch_size:32 num_blocks:1024 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][183/325] batch_size:32 num_blocks:1024 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][184/325] batch_size:32 num_blocks:1152 free_mem:22.54 GiB
INFO 01-31 03:37:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][184/325] batch_size:32 num_blocks:1152 free_mem:22.54 GiB
INFO 01-31 03:37:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][185/325] batch_size:32 num_blocks:1280 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][185/325] batch_size:32 num_blocks:1280 free_mem:22.54 GiB
INFO 01-31 03:37:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][186/325] batch_size:32 num_blocks:1408 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][186/325] batch_size:32 num_blocks:1408 free_mem:22.54 GiB
INFO 01-31 03:37:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][187/325] batch_size:32 num_blocks:1536 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][187/325] batch_size:32 num_blocks:1536 free_mem:22.54 GiB
INFO 01-31 03:37:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][188/325] batch_size:32 num_blocks:1664 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][188/325] batch_size:32 num_blocks:1664 free_mem:22.54 GiB
INFO 01-31 03:37:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][189/325] batch_size:32 num_blocks:1792 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][189/325] batch_size:32 num_blocks:1792 free_mem:22.54 GiB
INFO 01-31 03:37:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][190/325] batch_size:32 num_blocks:1920 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][190/325] batch_size:32 num_blocks:1920 free_mem:22.54 GiB
INFO 01-31 03:37:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][191/325] batch_size:32 num_blocks:2048 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][191/325] batch_size:32 num_blocks:2048 free_mem:22.54 GiB
INFO 01-31 03:37:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][192/325] batch_size:32 num_blocks:2176 free_mem:22.54 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][192/325] batch_size:32 num_blocks:2176 free_mem:22.54 GiB
INFO 01-31 03:37:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][193/325] batch_size:32 num_blocks:2304 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][193/325] batch_size:32 num_blocks:2304 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][194/325] batch_size:32 num_blocks:2432 free_mem:22.53 GiB
INFO 01-31 03:37:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][194/325] batch_size:32 num_blocks:2432 free_mem:22.53 GiB
INFO 01-31 03:37:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][195/325] batch_size:32 num_blocks:2560 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][195/325] batch_size:32 num_blocks:2560 free_mem:22.53 GiB
INFO 01-31 03:37:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][196/325] batch_size:32 num_blocks:2688 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][196/325] batch_size:32 num_blocks:2688 free_mem:22.53 GiB
INFO 01-31 03:37:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][197/325] batch_size:32 num_blocks:2816 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][197/325] batch_size:32 num_blocks:2816 free_mem:22.53 GiB
INFO 01-31 03:37:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][198/325] batch_size:32 num_blocks:2944 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][198/325] batch_size:32 num_blocks:2944 free_mem:22.53 GiB
INFO 01-31 03:37:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][199/325] batch_size:32 num_blocks:3072 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][199/325] batch_size:32 num_blocks:3072 free_mem:22.53 GiB
INFO 01-31 03:37:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][200/325] batch_size:32 num_blocks:3116 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][200/325] batch_size:32 num_blocks:3116 free_mem:22.53 GiB
INFO 01-31 03:37:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][201/325] batch_size:16 num_blocks:128 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][201/325] batch_size:16 num_blocks:128 free_mem:22.53 GiB
INFO 01-31 03:37:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][202/325] batch_size:16 num_blocks:256 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][202/325] batch_size:16 num_blocks:256 free_mem:22.53 GiB
INFO 01-31 03:37:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][203/325] batch_size:16 num_blocks:384 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][203/325] batch_size:16 num_blocks:384 free_mem:22.53 GiB
INFO 01-31 03:37:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][204/325] batch_size:16 num_blocks:512 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][204/325] batch_size:16 num_blocks:512 free_mem:22.53 GiB
INFO 01-31 03:37:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][205/325] batch_size:16 num_blocks:640 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][205/325] batch_size:16 num_blocks:640 free_mem:22.53 GiB
INFO 01-31 03:37:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][206/325] batch_size:16 num_blocks:768 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][206/325] batch_size:16 num_blocks:768 free_mem:22.53 GiB
INFO 01-31 03:37:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][207/325] batch_size:16 num_blocks:896 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:37:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][207/325] batch_size:16 num_blocks:896 free_mem:22.53 GiB
INFO 01-31 03:38:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][208/325] batch_size:16 num_blocks:1024 free_mem:22.53 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][208/325] batch_size:16 num_blocks:1024 free_mem:22.52 GiB
INFO 01-31 03:38:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][209/325] batch_size:16 num_blocks:1152 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][209/325] batch_size:16 num_blocks:1152 free_mem:22.52 GiB
INFO 01-31 03:38:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][210/325] batch_size:16 num_blocks:1280 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][210/325] batch_size:16 num_blocks:1280 free_mem:22.52 GiB
INFO 01-31 03:38:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][211/325] batch_size:16 num_blocks:1408 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][211/325] batch_size:16 num_blocks:1408 free_mem:22.52 GiB
INFO 01-31 03:38:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][212/325] batch_size:16 num_blocks:1536 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][212/325] batch_size:16 num_blocks:1536 free_mem:22.52 GiB
INFO 01-31 03:38:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][213/325] batch_size:16 num_blocks:1664 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][213/325] batch_size:16 num_blocks:1664 free_mem:22.52 GiB
INFO 01-31 03:38:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][214/325] batch_size:16 num_blocks:1792 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][214/325] batch_size:16 num_blocks:1792 free_mem:22.52 GiB
INFO 01-31 03:38:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][215/325] batch_size:16 num_blocks:1920 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][215/325] batch_size:16 num_blocks:1920 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][216/325] batch_size:16 num_blocks:2048 free_mem:22.52 GiB
INFO 01-31 03:38:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][216/325] batch_size:16 num_blocks:2048 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][217/325] batch_size:16 num_blocks:2176 free_mem:22.52 GiB
INFO 01-31 03:38:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][217/325] batch_size:16 num_blocks:2176 free_mem:22.52 GiB
INFO 01-31 03:38:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][218/325] batch_size:16 num_blocks:2304 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][218/325] batch_size:16 num_blocks:2304 free_mem:22.52 GiB
INFO 01-31 03:38:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][219/325] batch_size:16 num_blocks:2432 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][219/325] batch_size:16 num_blocks:2432 free_mem:22.52 GiB
INFO 01-31 03:38:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][220/325] batch_size:16 num_blocks:2560 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][220/325] batch_size:16 num_blocks:2560 free_mem:22.52 GiB
INFO 01-31 03:38:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][221/325] batch_size:16 num_blocks:2688 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][221/325] batch_size:16 num_blocks:2688 free_mem:22.52 GiB
INFO 01-31 03:38:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][222/325] batch_size:16 num_blocks:2816 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][222/325] batch_size:16 num_blocks:2816 free_mem:22.52 GiB
INFO 01-31 03:38:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][223/325] batch_size:16 num_blocks:2944 free_mem:22.52 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][223/325] batch_size:16 num_blocks:2944 free_mem:22.52 GiB
INFO 01-31 03:38:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][224/325] batch_size:16 num_blocks:3072 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][224/325] batch_size:16 num_blocks:3072 free_mem:22.51 GiB
INFO 01-31 03:38:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][225/325] batch_size:16 num_blocks:3116 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][225/325] batch_size:16 num_blocks:3116 free_mem:22.51 GiB
INFO 01-31 03:38:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][226/325] batch_size:8 num_blocks:128 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][226/325] batch_size:8 num_blocks:128 free_mem:22.51 GiB
INFO 01-31 03:38:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][227/325] batch_size:8 num_blocks:256 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][227/325] batch_size:8 num_blocks:256 free_mem:22.51 GiB
INFO 01-31 03:38:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][228/325] batch_size:8 num_blocks:384 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][228/325] batch_size:8 num_blocks:384 free_mem:22.51 GiB
INFO 01-31 03:38:16 hpu_model_runner.py:1473] [Warmup][Graph/Decode][229/325] batch_size:8 num_blocks:512 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:16 hpu_model_runner.py:1473] [Warmup][Graph/Decode][229/325] batch_size:8 num_blocks:512 free_mem:22.51 GiB
INFO 01-31 03:38:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][230/325] batch_size:8 num_blocks:640 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][230/325] batch_size:8 num_blocks:640 free_mem:22.51 GiB
INFO 01-31 03:38:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][231/325] batch_size:8 num_blocks:768 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][231/325] batch_size:8 num_blocks:768 free_mem:22.51 GiB
INFO 01-31 03:38:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][232/325] batch_size:8 num_blocks:896 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][232/325] batch_size:8 num_blocks:896 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][233/325] batch_size:8 num_blocks:1024 free_mem:22.51 GiB
INFO 01-31 03:38:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][233/325] batch_size:8 num_blocks:1024 free_mem:22.51 GiB
INFO 01-31 03:38:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][234/325] batch_size:8 num_blocks:1152 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][234/325] batch_size:8 num_blocks:1152 free_mem:22.51 GiB
INFO 01-31 03:38:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][235/325] batch_size:8 num_blocks:1280 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][235/325] batch_size:8 num_blocks:1280 free_mem:22.51 GiB
INFO 01-31 03:38:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][236/325] batch_size:8 num_blocks:1408 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][236/325] batch_size:8 num_blocks:1408 free_mem:22.51 GiB
INFO 01-31 03:38:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][237/325] batch_size:8 num_blocks:1536 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][237/325] batch_size:8 num_blocks:1536 free_mem:22.51 GiB
INFO 01-31 03:38:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][238/325] batch_size:8 num_blocks:1664 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][238/325] batch_size:8 num_blocks:1664 free_mem:22.51 GiB
INFO 01-31 03:38:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][239/325] batch_size:8 num_blocks:1792 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][239/325] batch_size:8 num_blocks:1792 free_mem:22.51 GiB
INFO 01-31 03:38:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][240/325] batch_size:8 num_blocks:1920 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][240/325] batch_size:8 num_blocks:1920 free_mem:22.51 GiB
INFO 01-31 03:38:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][241/325] batch_size:8 num_blocks:2048 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][241/325] batch_size:8 num_blocks:2048 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][242/325] batch_size:8 num_blocks:2176 free_mem:22.51 GiB
INFO 01-31 03:38:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][242/325] batch_size:8 num_blocks:2176 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][243/325] batch_size:8 num_blocks:2304 free_mem:22.51 GiB
INFO 01-31 03:38:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][243/325] batch_size:8 num_blocks:2304 free_mem:22.51 GiB
INFO 01-31 03:38:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][244/325] batch_size:8 num_blocks:2432 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][244/325] batch_size:8 num_blocks:2432 free_mem:22.51 GiB
INFO 01-31 03:38:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][245/325] batch_size:8 num_blocks:2560 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][245/325] batch_size:8 num_blocks:2560 free_mem:22.51 GiB
INFO 01-31 03:38:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][246/325] batch_size:8 num_blocks:2688 free_mem:22.51 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][246/325] batch_size:8 num_blocks:2688 free_mem:22.5 GiB
INFO 01-31 03:38:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][247/325] batch_size:8 num_blocks:2816 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][247/325] batch_size:8 num_blocks:2816 free_mem:22.5 GiB
INFO 01-31 03:38:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][248/325] batch_size:8 num_blocks:2944 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][248/325] batch_size:8 num_blocks:2944 free_mem:22.5 GiB
INFO 01-31 03:38:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][249/325] batch_size:8 num_blocks:3072 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][249/325] batch_size:8 num_blocks:3072 free_mem:22.5 GiB
INFO 01-31 03:38:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][250/325] batch_size:8 num_blocks:3116 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:32 hpu_model_runner.py:1473] [Warmup][Graph/Decode][250/325] batch_size:8 num_blocks:3116 free_mem:22.5 GiB
INFO 01-31 03:38:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][251/325] batch_size:4 num_blocks:128 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:33 hpu_model_runner.py:1473] [Warmup][Graph/Decode][251/325] batch_size:4 num_blocks:128 free_mem:22.5 GiB
INFO 01-31 03:38:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][252/325] batch_size:4 num_blocks:256 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][252/325] batch_size:4 num_blocks:256 free_mem:22.5 GiB
INFO 01-31 03:38:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][253/325] batch_size:4 num_blocks:384 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:34 hpu_model_runner.py:1473] [Warmup][Graph/Decode][253/325] batch_size:4 num_blocks:384 free_mem:22.5 GiB
INFO 01-31 03:38:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][254/325] batch_size:4 num_blocks:512 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:35 hpu_model_runner.py:1473] [Warmup][Graph/Decode][254/325] batch_size:4 num_blocks:512 free_mem:22.5 GiB
INFO 01-31 03:38:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][255/325] batch_size:4 num_blocks:640 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:36 hpu_model_runner.py:1473] [Warmup][Graph/Decode][255/325] batch_size:4 num_blocks:640 free_mem:22.5 GiB
INFO 01-31 03:38:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][256/325] batch_size:4 num_blocks:768 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:37 hpu_model_runner.py:1473] [Warmup][Graph/Decode][256/325] batch_size:4 num_blocks:768 free_mem:22.5 GiB
INFO 01-31 03:38:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][257/325] batch_size:4 num_blocks:896 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][257/325] batch_size:4 num_blocks:896 free_mem:22.5 GiB
INFO 01-31 03:38:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][258/325] batch_size:4 num_blocks:1024 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:38 hpu_model_runner.py:1473] [Warmup][Graph/Decode][258/325] batch_size:4 num_blocks:1024 free_mem:22.5 GiB
INFO 01-31 03:38:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][259/325] batch_size:4 num_blocks:1152 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:39 hpu_model_runner.py:1473] [Warmup][Graph/Decode][259/325] batch_size:4 num_blocks:1152 free_mem:22.5 GiB
INFO 01-31 03:38:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][260/325] batch_size:4 num_blocks:1280 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:40 hpu_model_runner.py:1473] [Warmup][Graph/Decode][260/325] batch_size:4 num_blocks:1280 free_mem:22.5 GiB
INFO 01-31 03:38:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][261/325] batch_size:4 num_blocks:1408 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][261/325] batch_size:4 num_blocks:1408 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][262/325] batch_size:4 num_blocks:1536 free_mem:22.5 GiB
INFO 01-31 03:38:41 hpu_model_runner.py:1473] [Warmup][Graph/Decode][262/325] batch_size:4 num_blocks:1536 free_mem:22.5 GiB
INFO 01-31 03:38:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][263/325] batch_size:4 num_blocks:1664 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:42 hpu_model_runner.py:1473] [Warmup][Graph/Decode][263/325] batch_size:4 num_blocks:1664 free_mem:22.5 GiB
INFO 01-31 03:38:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][264/325] batch_size:4 num_blocks:1792 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:43 hpu_model_runner.py:1473] [Warmup][Graph/Decode][264/325] batch_size:4 num_blocks:1792 free_mem:22.5 GiB
INFO 01-31 03:38:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][265/325] batch_size:4 num_blocks:1920 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][265/325] batch_size:4 num_blocks:1920 free_mem:22.5 GiB
INFO 01-31 03:38:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][266/325] batch_size:4 num_blocks:2048 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:44 hpu_model_runner.py:1473] [Warmup][Graph/Decode][266/325] batch_size:4 num_blocks:2048 free_mem:22.5 GiB
INFO 01-31 03:38:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][267/325] batch_size:4 num_blocks:2176 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:45 hpu_model_runner.py:1473] [Warmup][Graph/Decode][267/325] batch_size:4 num_blocks:2176 free_mem:22.5 GiB
INFO 01-31 03:38:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][268/325] batch_size:4 num_blocks:2304 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:46 hpu_model_runner.py:1473] [Warmup][Graph/Decode][268/325] batch_size:4 num_blocks:2304 free_mem:22.5 GiB
INFO 01-31 03:38:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][269/325] batch_size:4 num_blocks:2432 free_mem:22.5 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:47 hpu_model_runner.py:1473] [Warmup][Graph/Decode][269/325] batch_size:4 num_blocks:2432 free_mem:22.49 GiB
INFO 01-31 03:38:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][270/325] batch_size:4 num_blocks:2560 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][270/325] batch_size:4 num_blocks:2560 free_mem:22.49 GiB
INFO 01-31 03:38:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][271/325] batch_size:4 num_blocks:2688 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:48 hpu_model_runner.py:1473] [Warmup][Graph/Decode][271/325] batch_size:4 num_blocks:2688 free_mem:22.49 GiB
INFO 01-31 03:38:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][272/325] batch_size:4 num_blocks:2816 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:49 hpu_model_runner.py:1473] [Warmup][Graph/Decode][272/325] batch_size:4 num_blocks:2816 free_mem:22.49 GiB
INFO 01-31 03:38:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][273/325] batch_size:4 num_blocks:2944 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:50 hpu_model_runner.py:1473] [Warmup][Graph/Decode][273/325] batch_size:4 num_blocks:2944 free_mem:22.49 GiB
INFO 01-31 03:38:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][274/325] batch_size:4 num_blocks:3072 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:51 hpu_model_runner.py:1473] [Warmup][Graph/Decode][274/325] batch_size:4 num_blocks:3072 free_mem:22.49 GiB
INFO 01-31 03:38:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][275/325] batch_size:4 num_blocks:3116 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][275/325] batch_size:4 num_blocks:3116 free_mem:22.49 GiB
INFO 01-31 03:38:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][276/325] batch_size:2 num_blocks:128 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:52 hpu_model_runner.py:1473] [Warmup][Graph/Decode][276/325] batch_size:2 num_blocks:128 free_mem:22.49 GiB
INFO 01-31 03:38:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][277/325] batch_size:2 num_blocks:256 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:53 hpu_model_runner.py:1473] [Warmup][Graph/Decode][277/325] batch_size:2 num_blocks:256 free_mem:22.49 GiB
INFO 01-31 03:38:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][278/325] batch_size:2 num_blocks:384 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:54 hpu_model_runner.py:1473] [Warmup][Graph/Decode][278/325] batch_size:2 num_blocks:384 free_mem:22.49 GiB
INFO 01-31 03:38:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][279/325] batch_size:2 num_blocks:512 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][279/325] batch_size:2 num_blocks:512 free_mem:22.49 GiB
INFO 01-31 03:38:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][280/325] batch_size:2 num_blocks:640 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:55 hpu_model_runner.py:1473] [Warmup][Graph/Decode][280/325] batch_size:2 num_blocks:640 free_mem:22.49 GiB
INFO 01-31 03:38:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][281/325] batch_size:2 num_blocks:768 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:56 hpu_model_runner.py:1473] [Warmup][Graph/Decode][281/325] batch_size:2 num_blocks:768 free_mem:22.49 GiB
INFO 01-31 03:38:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][282/325] batch_size:2 num_blocks:896 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:57 hpu_model_runner.py:1473] [Warmup][Graph/Decode][282/325] batch_size:2 num_blocks:896 free_mem:22.49 GiB
INFO 01-31 03:38:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][283/325] batch_size:2 num_blocks:1024 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:58 hpu_model_runner.py:1473] [Warmup][Graph/Decode][283/325] batch_size:2 num_blocks:1024 free_mem:22.49 GiB
INFO 01-31 03:38:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][284/325] batch_size:2 num_blocks:1152 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][284/325] batch_size:2 num_blocks:1152 free_mem:22.49 GiB
INFO 01-31 03:38:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][285/325] batch_size:2 num_blocks:1280 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:38:59 hpu_model_runner.py:1473] [Warmup][Graph/Decode][285/325] batch_size:2 num_blocks:1280 free_mem:22.49 GiB
INFO 01-31 03:39:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][286/325] batch_size:2 num_blocks:1408 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:00 hpu_model_runner.py:1473] [Warmup][Graph/Decode][286/325] batch_size:2 num_blocks:1408 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][287/325] batch_size:2 num_blocks:1536 free_mem:22.49 GiB
INFO 01-31 03:39:01 hpu_model_runner.py:1473] [Warmup][Graph/Decode][287/325] batch_size:2 num_blocks:1536 free_mem:22.49 GiB
INFO 01-31 03:39:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][288/325] batch_size:2 num_blocks:1664 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:02 hpu_model_runner.py:1473] [Warmup][Graph/Decode][288/325] batch_size:2 num_blocks:1664 free_mem:22.49 GiB
INFO 01-31 03:39:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][289/325] batch_size:2 num_blocks:1792 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][289/325] batch_size:2 num_blocks:1792 free_mem:22.49 GiB
INFO 01-31 03:39:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][290/325] batch_size:2 num_blocks:1920 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:03 hpu_model_runner.py:1473] [Warmup][Graph/Decode][290/325] batch_size:2 num_blocks:1920 free_mem:22.49 GiB
INFO 01-31 03:39:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][291/325] batch_size:2 num_blocks:2048 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:04 hpu_model_runner.py:1473] [Warmup][Graph/Decode][291/325] batch_size:2 num_blocks:2048 free_mem:22.49 GiB
INFO 01-31 03:39:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][292/325] batch_size:2 num_blocks:2176 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:05 hpu_model_runner.py:1473] [Warmup][Graph/Decode][292/325] batch_size:2 num_blocks:2176 free_mem:22.49 GiB
INFO 01-31 03:39:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][293/325] batch_size:2 num_blocks:2304 free_mem:22.49 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:06 hpu_model_runner.py:1473] [Warmup][Graph/Decode][293/325] batch_size:2 num_blocks:2304 free_mem:22.48 GiB
INFO 01-31 03:39:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][294/325] batch_size:2 num_blocks:2432 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][294/325] batch_size:2 num_blocks:2432 free_mem:22.48 GiB
INFO 01-31 03:39:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][295/325] batch_size:2 num_blocks:2560 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:07 hpu_model_runner.py:1473] [Warmup][Graph/Decode][295/325] batch_size:2 num_blocks:2560 free_mem:22.48 GiB
INFO 01-31 03:39:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][296/325] batch_size:2 num_blocks:2688 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:08 hpu_model_runner.py:1473] [Warmup][Graph/Decode][296/325] batch_size:2 num_blocks:2688 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][297/325] batch_size:2 num_blocks:2816 free_mem:22.48 GiB
INFO 01-31 03:39:09 hpu_model_runner.py:1473] [Warmup][Graph/Decode][297/325] batch_size:2 num_blocks:2816 free_mem:22.48 GiB
INFO 01-31 03:39:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][298/325] batch_size:2 num_blocks:2944 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:10 hpu_model_runner.py:1473] [Warmup][Graph/Decode][298/325] batch_size:2 num_blocks:2944 free_mem:22.48 GiB
INFO 01-31 03:39:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][299/325] batch_size:2 num_blocks:3072 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][299/325] batch_size:2 num_blocks:3072 free_mem:22.48 GiB
INFO 01-31 03:39:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][300/325] batch_size:2 num_blocks:3116 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:11 hpu_model_runner.py:1473] [Warmup][Graph/Decode][300/325] batch_size:2 num_blocks:3116 free_mem:22.48 GiB
INFO 01-31 03:39:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][301/325] batch_size:1 num_blocks:128 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:12 hpu_model_runner.py:1473] [Warmup][Graph/Decode][301/325] batch_size:1 num_blocks:128 free_mem:22.48 GiB
INFO 01-31 03:39:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][302/325] batch_size:1 num_blocks:256 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:13 hpu_model_runner.py:1473] [Warmup][Graph/Decode][302/325] batch_size:1 num_blocks:256 free_mem:22.48 GiB
INFO 01-31 03:39:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][303/325] batch_size:1 num_blocks:384 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][303/325] batch_size:1 num_blocks:384 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][304/325] batch_size:1 num_blocks:512 free_mem:22.48 GiB
INFO 01-31 03:39:14 hpu_model_runner.py:1473] [Warmup][Graph/Decode][304/325] batch_size:1 num_blocks:512 free_mem:22.48 GiB
INFO 01-31 03:39:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][305/325] batch_size:1 num_blocks:640 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:15 hpu_model_runner.py:1473] [Warmup][Graph/Decode][305/325] batch_size:1 num_blocks:640 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:16 hpu_model_runner.py:1473] [Warmup][Graph/Decode][306/325] batch_size:1 num_blocks:768 free_mem:22.48 GiB
INFO 01-31 03:39:16 hpu_model_runner.py:1473] [Warmup][Graph/Decode][306/325] batch_size:1 num_blocks:768 free_mem:22.48 GiB
INFO 01-31 03:39:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][307/325] batch_size:1 num_blocks:896 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:17 hpu_model_runner.py:1473] [Warmup][Graph/Decode][307/325] batch_size:1 num_blocks:896 free_mem:22.48 GiB
INFO 01-31 03:39:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][308/325] batch_size:1 num_blocks:1024 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][308/325] batch_size:1 num_blocks:1024 free_mem:22.48 GiB
INFO 01-31 03:39:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][309/325] batch_size:1 num_blocks:1152 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:18 hpu_model_runner.py:1473] [Warmup][Graph/Decode][309/325] batch_size:1 num_blocks:1152 free_mem:22.48 GiB
INFO 01-31 03:39:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][310/325] batch_size:1 num_blocks:1280 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:19 hpu_model_runner.py:1473] [Warmup][Graph/Decode][310/325] batch_size:1 num_blocks:1280 free_mem:22.48 GiB
INFO 01-31 03:39:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][311/325] batch_size:1 num_blocks:1408 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:20 hpu_model_runner.py:1473] [Warmup][Graph/Decode][311/325] batch_size:1 num_blocks:1408 free_mem:22.48 GiB
INFO 01-31 03:39:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][312/325] batch_size:1 num_blocks:1536 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][312/325] batch_size:1 num_blocks:1536 free_mem:22.48 GiB
INFO 01-31 03:39:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][313/325] batch_size:1 num_blocks:1664 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:21 hpu_model_runner.py:1473] [Warmup][Graph/Decode][313/325] batch_size:1 num_blocks:1664 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][314/325] batch_size:1 num_blocks:1792 free_mem:22.48 GiB
INFO 01-31 03:39:22 hpu_model_runner.py:1473] [Warmup][Graph/Decode][314/325] batch_size:1 num_blocks:1792 free_mem:22.48 GiB
INFO 01-31 03:39:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][315/325] batch_size:1 num_blocks:1920 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:23 hpu_model_runner.py:1473] [Warmup][Graph/Decode][315/325] batch_size:1 num_blocks:1920 free_mem:22.48 GiB
INFO 01-31 03:39:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][316/325] batch_size:1 num_blocks:2048 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:24 hpu_model_runner.py:1473] [Warmup][Graph/Decode][316/325] batch_size:1 num_blocks:2048 free_mem:22.48 GiB
INFO 01-31 03:39:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][317/325] batch_size:1 num_blocks:2176 free_mem:22.48 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][317/325] batch_size:1 num_blocks:2176 free_mem:22.47 GiB
INFO 01-31 03:39:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][318/325] batch_size:1 num_blocks:2304 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:25 hpu_model_runner.py:1473] [Warmup][Graph/Decode][318/325] batch_size:1 num_blocks:2304 free_mem:22.47 GiB
INFO 01-31 03:39:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][319/325] batch_size:1 num_blocks:2432 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:26 hpu_model_runner.py:1473] [Warmup][Graph/Decode][319/325] batch_size:1 num_blocks:2432 free_mem:22.47 GiB
INFO 01-31 03:39:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][320/325] batch_size:1 num_blocks:2560 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:27 hpu_model_runner.py:1473] [Warmup][Graph/Decode][320/325] batch_size:1 num_blocks:2560 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][321/325] batch_size:1 num_blocks:2688 free_mem:22.47 GiB
INFO 01-31 03:39:28 hpu_model_runner.py:1473] [Warmup][Graph/Decode][321/325] batch_size:1 num_blocks:2688 free_mem:22.47 GiB
INFO 01-31 03:39:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][322/325] batch_size:1 num_blocks:2816 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][322/325] batch_size:1 num_blocks:2816 free_mem:22.47 GiB
INFO 01-31 03:39:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][323/325] batch_size:1 num_blocks:2944 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:29 hpu_model_runner.py:1473] [Warmup][Graph/Decode][323/325] batch_size:1 num_blocks:2944 free_mem:22.47 GiB
INFO 01-31 03:39:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][324/325] batch_size:1 num_blocks:3072 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:30 hpu_model_runner.py:1473] [Warmup][Graph/Decode][324/325] batch_size:1 num_blocks:3072 free_mem:22.47 GiB
INFO 01-31 03:39:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][325/325] batch_size:1 num_blocks:3116 free_mem:22.47 GiB
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:31 hpu_model_runner.py:1473] [Warmup][Graph/Decode][325/325] batch_size:1 num_blocks:3116 free_mem:22.47 GiB
INFO 01-31 03:39:32 hpu_model_runner.py:1544] Graph/Prompt captured:31 (100.0%) used_mem:1.188 MiB buckets:[(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (8, 128), (8, 256), (8, 384), (8, 512), (16, 128), (16, 256), (32, 128)]
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:32 hpu_model_runner.py:1544] Graph/Prompt captured:31 (100.0%) used_mem:1.188 MiB buckets:[(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (8, 128), (8, 256), (8, 384), (8, 512), (16, 128), (16, 256), (32, 128)]
INFO 01-31 03:39:32 hpu_model_runner.py:1544] Graph/Decode captured:325 (100.0%) used_mem:397.9 MiB buckets:[(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (1, 1152), (1, 1280), (1, 1408), (1, 1536), (1, 1664), (1, 1792), (1, 1920), (1, 2048), (1, 2176), (1, 2304), (1, 2432), (1, 2560), (1, 2688), (1, 2816), (1, 2944), (1, 3072), (1, 3116), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (2, 1152), (2, 1280), (2, 1408), (2, 1536), (2, 1664), (2, 1792), (2, 1920), (2, 2048), (2, 2176), (2, 2304), (2, 2432), (2, 2560), (2, 2688), (2, 2816), (2, 2944), (2, 3072), (2, 3116), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (4, 1152), (4, 1280), (4, 1408), (4, 1536), (4, 1664), (4, 1792), (4, 1920), (4, 2048), (4, 2176), (4, 2304), (4, 2432), (4, 2560), (4, 2688), (4, 2816), (4, 2944), (4, 3072), (4, 3116), (8, 128), (8, 256), (8, 384), (8, 512), (8, 640), (8, 768), (8, 896), (8, 1024), (8, 1152), (8, 1280), (8, 1408), (8, 1536), (8, 1664), (8, 1792), (8, 1920), (8, 2048), (8, 2176), (8, 2304), (8, 2432), (8, 2560), (8, 2688), (8, 2816), (8, 2944), (8, 3072), (8, 3116), (16, 128), (16, 256), (16, 384), (16, 512), (16, 640), (16, 768), (16, 896), (16, 1024), (16, 1152), (16, 1280), (16, 1408), (16, 1536), (16, 1664), (16, 1792), (16, 1920), (16, 2048), (16, 2176), (16, 2304), (16, 2432), (16, 2560), (16, 2688), (16, 2816), (16, 2944), (16, 3072), (16, 3116), (32, 128), (32, 256), (32, 384), (32, 512), (32, 640), (32, 768), (32, 896), (32, 1024), (32, 1152), (32, 1280), (32, 1408), (32, 1536), (32, 1664), (32, 1792), (32, 1920), (32, 2048), (32, 2176), (32, 2304), (32, 2432), (32, 2560), (32, 2688), (32, 2816), (32, 2944), (32, 3072), (32, 3116), (64, 128), (64, 256), (64, 384), (64, 512), (64, 640), (64, 768), (64, 896), (64, 1024), (64, 1152), (64, 1280), (64, 1408), (64, 1536), (64, 1664), (64, 1792), (64, 1920), (64, 2048), (64, 2176), (64, 2304), (64, 2432), (64, 2560), (64, 2688), (64, 2816), (64, 2944), (64, 3072), (64, 3116), (96, 128), (96, 256), (96, 384), (96, 512), (96, 640), (96, 768), (96, 896), (96, 1024), (96, 1152), (96, 1280), (96, 1408), (96, 1536), (96, 1664), (96, 1792), (96, 1920), (96, 2048), (96, 2176), (96, 2304), (96, 2432), (96, 2560), (96, 2688), (96, 2816), (96, 2944), (96, 3072), (96, 3116), (128, 128), (128, 256), (128, 384), (128, 512), (128, 640), (128, 768), (128, 896), (128, 1024), (128, 1152), (128, 1280), (128, 1408), (128, 1536), (128, 1664), (128, 1792), (128, 1920), (128, 2048), (128, 2176), (128, 2304), (128, 2432), (128, 2560), (128, 2688), (128, 2816), (128, 2944), (128, 3072), (128, 3116), (160, 128), (160, 256), (160, 384), (160, 512), (160, 640), (160, 768), (160, 896), (160, 1024), (160, 1152), (160, 1280), (160, 1408), (160, 1536), (160, 1664), (160, 1792), (160, 1920), (160, 2048), (160, 2176), (160, 2304), (160, 2432), (160, 2560), (160, 2688), (160, 2816), (160, 2944), (160, 3072), (160, 3116), (192, 128), (192, 256), (192, 384), (192, 512), (192, 640), (192, 768), (192, 896), (192, 1024), (192, 1152), (192, 1280), (192, 1408), (192, 1536), (192, 1664), (192, 1792), (192, 1920), (192, 2048), (192, 2176), (192, 2304), (192, 2432), (192, 2560), (192, 2688), (192, 2816), (192, 2944), (192, 3072), (192, 3116), (224, 128), (224, 256), (224, 384), (224, 512), (224, 640), (224, 768), (224, 896), (224, 1024), (224, 1152), (224, 1280), (224, 1408), (224, 1536), (224, 1664), (224, 1792), (224, 1920), (224, 2048), (224, 2176), (224, 2304), (224, 2432), (224, 2560), (224, 2688), (224, 2816), (224, 2944), (224, 3072), (224, 3116), (256, 128), (256, 256), (256, 384), (256, 512), (256, 640), (256, 768), (256, 896), (256, 1024), (256, 1152), (256, 1280), (256, 1408), (256, 1536), (256, 1664), (256, 1792), (256, 1920), (256, 2048), (256, 2176), (256, 2304), (256, 2432), (256, 2560), (256, 2688), (256, 2816), (256, 2944), (256, 3072), (256, 3116)]
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:32 hpu_model_runner.py:1544] Graph/Decode captured:325 (100.0%) used_mem:397.9 MiB buckets:[(1, 128), (1, 256), (1, 384), (1, 512), (1, 640), (1, 768), (1, 896), (1, 1024), (1, 1152), (1, 1280), (1, 1408), (1, 1536), (1, 1664), (1, 1792), (1, 1920), (1, 2048), (1, 2176), (1, 2304), (1, 2432), (1, 2560), (1, 2688), (1, 2816), (1, 2944), (1, 3072), (1, 3116), (2, 128), (2, 256), (2, 384), (2, 512), (2, 640), (2, 768), (2, 896), (2, 1024), (2, 1152), (2, 1280), (2, 1408), (2, 1536), (2, 1664), (2, 1792), (2, 1920), (2, 2048), (2, 2176), (2, 2304), (2, 2432), (2, 2560), (2, 2688), (2, 2816), (2, 2944), (2, 3072), (2, 3116), (4, 128), (4, 256), (4, 384), (4, 512), (4, 640), (4, 768), (4, 896), (4, 1024), (4, 1152), (4, 1280), (4, 1408), (4, 1536), (4, 1664), (4, 1792), (4, 1920), (4, 2048), (4, 2176), (4, 2304), (4, 2432), (4, 2560), (4, 2688), (4, 2816), (4, 2944), (4, 3072), (4, 3116), (8, 128), (8, 256), (8, 384), (8, 512), (8, 640), (8, 768), (8, 896), (8, 1024), (8, 1152), (8, 1280), (8, 1408), (8, 1536), (8, 1664), (8, 1792), (8, 1920), (8, 2048), (8, 2176), (8, 2304), (8, 2432), (8, 2560), (8, 2688), (8, 2816), (8, 2944), (8, 3072), (8, 3116), (16, 128), (16, 256), (16, 384), (16, 512), (16, 640), (16, 768), (16, 896), (16, 1024), (16, 1152), (16, 1280), (16, 1408), (16, 1536), (16, 1664), (16, 1792), (16, 1920), (16, 2048), (16, 2176), (16, 2304), (16, 2432), (16, 2560), (16, 2688), (16, 2816), (16, 2944), (16, 3072), (16, 3116), (32, 128), (32, 256), (32, 384), (32, 512), (32, 640), (32, 768), (32, 896), (32, 1024), (32, 1152), (32, 1280), (32, 1408), (32, 1536), (32, 1664), (32, 1792), (32, 1920), (32, 2048), (32, 2176), (32, 2304), (32, 2432), (32, 2560), (32, 2688), (32, 2816), (32, 2944), (32, 3072), (32, 3116), (64, 128), (64, 256), (64, 384), (64, 512), (64, 640), (64, 768), (64, 896), (64, 1024), (64, 1152), (64, 1280), (64, 1408), (64, 1536), (64, 1664), (64, 1792), (64, 1920), (64, 2048), (64, 2176), (64, 2304), (64, 2432), (64, 2560), (64, 2688), (64, 2816), (64, 2944), (64, 3072), (64, 3116), (96, 128), (96, 256), (96, 384), (96, 512), (96, 640), (96, 768), (96, 896), (96, 1024), (96, 1152), (96, 1280), (96, 1408), (96, 1536), (96, 1664), (96, 1792), (96, 1920), (96, 2048), (96, 2176), (96, 2304), (96, 2432), (96, 2560), (96, 2688), (96, 2816), (96, 2944), (96, 3072), (96, 3116), (128, 128), (128, 256), (128, 384), (128, 512), (128, 640), (128, 768), (128, 896), (128, 1024), (128, 1152), (128, 1280), (128, 1408), (128, 1536), (128, 1664), (128, 1792), (128, 1920), (128, 2048), (128, 2176), (128, 2304), (128, 2432), (128, 2560), (128, 2688), (128, 2816), (128, 2944), (128, 3072), (128, 3116), (160, 128), (160, 256), (160, 384), (160, 512), (160, 640), (160, 768), (160, 896), (160, 1024), (160, 1152), (160, 1280), (160, 1408), (160, 1536), (160, 1664), (160, 1792), (160, 1920), (160, 2048), (160, 2176), (160, 2304), (160, 2432), (160, 2560), (160, 2688), (160, 2816), (160, 2944), (160, 3072), (160, 3116), (192, 128), (192, 256), (192, 384), (192, 512), (192, 640), (192, 768), (192, 896), (192, 1024), (192, 1152), (192, 1280), (192, 1408), (192, 1536), (192, 1664), (192, 1792), (192, 1920), (192, 2048), (192, 2176), (192, 2304), (192, 2432), (192, 2560), (192, 2688), (192, 2816), (192, 2944), (192, 3072), (192, 3116), (224, 128), (224, 256), (224, 384), (224, 512), (224, 640), (224, 768), (224, 896), (224, 1024), (224, 1152), (224, 1280), (224, 1408), (224, 1536), (224, 1664), (224, 1792), (224, 1920), (224, 2048), (224, 2176), (224, 2304), (224, 2432), (224, 2560), (224, 2688), (224, 2816), (224, 2944), (224, 3072), (224, 3116), (256, 128), (256, 256), (256, 384), (256, 512), (256, 640), (256, 768), (256, 896), (256, 1024), (256, 1152), (256, 1280), (256, 1408), (256, 1536), (256, 1664), (256, 1792), (256, 1920), (256, 2048), (256, 2176), (256, 2304), (256, 2432), (256, 2560), (256, 2688), (256, 2816), (256, 2944), (256, 3072), (256, 3116)]
INFO 01-31 03:39:32 hpu_model_runner.py:1670] Warmup finished in 1090 secs, allocated 398.3 MiB of device memory
(VllmWorkerProcess pid=19699) INFO 01-31 03:39:32 hpu_model_runner.py:1670] Warmup finished in 1090 secs, allocated 398.7 MiB of device memory
INFO 01-31 03:39:32 api_server.py:250] vLLM to use /tmp/tmpzfa9k98y as PROMETHEUS_MULTIPROC_DIR
INFO 01-31 03:39:32 api_server.py:534] Using supplied chat template:
INFO 01-31 03:39:32 api_server.py:534] None
INFO 01-31 03:39:32 launcher.py:19] Available routes are:
INFO 01-31 03:39:32 launcher.py:27] Route: /openapi.json, Methods: HEAD, GET
INFO 01-31 03:39:32 launcher.py:27] Route: /docs, Methods: HEAD, GET
INFO 01-31 03:39:32 launcher.py:27] Route: /docs/oauth2-redirect, Methods: HEAD, GET
INFO 01-31 03:39:32 launcher.py:27] Route: /redoc, Methods: HEAD, GET
INFO 01-31 03:39:32 launcher.py:27] Route: /health, Methods: GET
INFO 01-31 03:39:32 launcher.py:27] Route: /tokenize, Methods: POST
INFO 01-31 03:39:32 launcher.py:27] Route: /detokenize, Methods: POST
INFO 01-31 03:39:32 launcher.py:27] Route: /v1/models, Methods: GET
INFO 01-31 03:39:32 launcher.py:27] Route: /version, Methods: GET
INFO 01-31 03:39:32 launcher.py:27] Route: /v1/chat/completions, Methods: POST
INFO 01-31 03:39:32 launcher.py:27] Route: /v1/completions, Methods: POST
INFO 01-31 03:39:32 launcher.py:27] Route: /v1/embeddings, Methods: POST
INFO:     Started server process [19463]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)
INFO 01-31 03:39:42 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 01-31 03:39:52 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 01-31 03:40:02 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 01-31 03:40:12 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 01-31 03:40:22 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 01-31 03:40:32 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 01-31 03:40:42 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO:     127.0.0.1:48438 - "GET /v1/models HTTP/1.1" 200 OK
INFO 01-31 03:40:52 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.
INFO 01-31 03:40:52 chat_utils.py:331] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
thread '<unnamed>' panicked at /mount/work-dir/tokenizers-0.21.0/tokenizers-0.21.0/vendor/rayon-core/src/registry.rs:168:10:
The global thread pool has not been initialized.: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 179, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 491, in add_request_id
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 155, in call_next
    raise RuntimeError("No response returned.")
RuntimeError: No response returned.

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/opt/app-root/lib64/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
  |     result = await app(  # type: ignore[func-returns-value]
  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/app-root/lib64/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
  |     return await self.app(scope, receive, send)
  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
  |     await super().__call__(scope, receive, send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/applications.py", line 112, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
  |     await self.app(scope, receive, _send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 178, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/_backends/_asyncio.py", line 767, in __aexit__
  |     raise BaseExceptionGroup(
  | BaseExceptionGroup: unhandled errors in a TaskGroup (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 111, in receive
    |     return self.receive_nowait()
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 106, in receive_nowait
    |     raise WouldBlock
    | anyio.WouldBlock
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 124, in receive
    |     return receiver.item
    |            ^^^^^^^^^^^^^
    | AttributeError: 'MemoryObjectItemReceiver' object has no attribute 'item'
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 148, in call_next
    |     message = await recv_stream.receive()
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 126, in receive
    |     raise EndOfStream
    | anyio.EndOfStream
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 179, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 491, in add_request_id
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 155, in call_next
    |     raise RuntimeError("No response returned.")
    | RuntimeError: No response returned.
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 141, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
    |     await self.app(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 715, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 735, in app
    |     await route.handle(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 288, in handle
    |     await self.app(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 76, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 73, in app
    |     response = await f(request)
    |                ^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/routing.py", line 301, in app
    |     raw_response = await run_endpoint_function(
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    |     return await dependant.call(**values)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 348, in create_chat_completion
    |     generator = await handler.create_chat_completion(request, raw_request)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_chat.py", line 160, in create_chat_completion
    |     ) = await self._preprocess_chat(
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 496, in _preprocess_chat
    |     prompt_inputs = self._tokenize_prompt_input(
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 326, in _tokenize_prompt_input
    |     return next(
    |            ^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 349, in _tokenize_prompt_inputs
    |     yield self._normalize_prompt_text_to_input(
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 234, in _normalize_prompt_text_to_input
    |     encoded = tokenizer(prompt, add_special_tokens=add_special_tokens)
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2868, in __call__
    |     encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2978, in _call_one
    |     return self.encode_plus(
    |            ^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3054, in encode_plus
    |     return self._encode_plus(
    |            ^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 613, in _encode_plus
    |     batched_output = self._batch_encode_plus(
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 539, in _batch_encode_plus
    |     encodings = self._tokenizer.encode_batch(
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | pyo3_runtime.PanicException: The global thread pool has not been initialized.: ThreadPoolBuildError { kind: IOError(Os { code: 11, kind: WouldBlock, message: "Resource temporarily unavailable" }) }
    +------------------------------------
INFO:     127.0.0.1:60148 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
thread '<unnamed>' panicked at /mount/work-dir/tokenizers-0.21.0/tokenizers-0.21.0/vendor/rayon-core/src/registry.rs:168:10:
The global thread pool has not been initialized.: ThreadPoolBuildError { kind: GlobalPoolAlreadyInitialized }
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 179, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 491, in add_request_id
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 155, in call_next
    raise RuntimeError("No response returned.")
RuntimeError: No response returned.

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/opt/app-root/lib64/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
  |     result = await app(  # type: ignore[func-returns-value]
  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/app-root/lib64/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
  |     return await self.app(scope, receive, send)
  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
  |     await super().__call__(scope, receive, send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/applications.py", line 112, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
  |     await self.app(scope, receive, _send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 178, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/_backends/_asyncio.py", line 767, in __aexit__
  |     raise BaseExceptionGroup(
  | BaseExceptionGroup: unhandled errors in a TaskGroup (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 111, in receive
    |     return self.receive_nowait()
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 106, in receive_nowait
    |     raise WouldBlock
    | anyio.WouldBlock
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 124, in receive
    |     return receiver.item
    |            ^^^^^^^^^^^^^
    | AttributeError: 'MemoryObjectItemReceiver' object has no attribute 'item'
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 148, in call_next
    |     message = await recv_stream.receive()
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 126, in receive
    |     raise EndOfStream
    | anyio.EndOfStream
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 179, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 491, in add_request_id
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 155, in call_next
    |     raise RuntimeError("No response returned.")
    | RuntimeError: No response returned.
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 141, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
    |     await self.app(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 715, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 735, in app
    |     await route.handle(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 288, in handle
    |     await self.app(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 76, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 73, in app
    |     response = await f(request)
    |                ^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/routing.py", line 301, in app
    |     raw_response = await run_endpoint_function(
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    |     return await dependant.call(**values)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 348, in create_chat_completion
    |     generator = await handler.create_chat_completion(request, raw_request)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_chat.py", line 160, in create_chat_completion
    |     ) = await self._preprocess_chat(
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 496, in _preprocess_chat
    |     prompt_inputs = self._tokenize_prompt_input(
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 326, in _tokenize_prompt_input
    |     return next(
    |            ^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 349, in _tokenize_prompt_inputs
    |     yield self._normalize_prompt_text_to_input(
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 234, in _normalize_prompt_text_to_input
    |     encoded = tokenizer(prompt, add_special_tokens=add_special_tokens)
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2868, in __call__
    |     encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2978, in _call_one
    |     return self.encode_plus(
    |            ^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3054, in encode_plus
    |     return self._encode_plus(
    |            ^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 613, in _encode_plus
    |     batched_output = self._batch_encode_plus(
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 539, in _batch_encode_plus
    |     encodings = self._tokenizer.encode_batch(
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | pyo3_runtime.PanicException: The global thread pool has not been initialized.: ThreadPoolBuildError { kind: GlobalPoolAlreadyInitialized }
    +------------------------------------
INFO:     127.0.0.1:60150 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
thread '<unnamed>' panicked at /mount/work-dir/tokenizers-0.21.0/tokenizers-0.21.0/vendor/rayon-core/src/registry.rs:168:10:
The global thread pool has not been initialized.: ThreadPoolBuildError { kind: GlobalPoolAlreadyInitialized }
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 179, in __call__
    response = await self.dispatch_func(request, call_next)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 491, in add_request_id
    response = await call_next(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 155, in call_next
    raise RuntimeError("No response returned.")
RuntimeError: No response returned.

During handling of the above exception, another exception occurred:

  + Exception Group Traceback (most recent call last):
  |   File "/opt/app-root/lib64/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 409, in run_asgi
  |     result = await app(  # type: ignore[func-returns-value]
  |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/app-root/lib64/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
  |     return await self.app(scope, receive, send)
  |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/applications.py", line 1054, in __call__
  |     await super().__call__(scope, receive, send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/applications.py", line 112, in __call__
  |     await self.middleware_stack(scope, receive, send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/errors.py", line 165, in __call__
  |     await self.app(scope, receive, _send)
  |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 178, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/_backends/_asyncio.py", line 767, in __aexit__
  |     raise BaseExceptionGroup(
  | BaseExceptionGroup: unhandled errors in a TaskGroup (2 sub-exceptions)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 111, in receive
    |     return self.receive_nowait()
    |            ^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 106, in receive_nowait
    |     raise WouldBlock
    | anyio.WouldBlock
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 124, in receive
    |     return receiver.item
    |            ^^^^^^^^^^^^^
    | AttributeError: 'MemoryObjectItemReceiver' object has no attribute 'item'
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 148, in call_next
    |     message = await recv_stream.receive()
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/anyio/streams/memory.py", line 126, in receive
    |     raise EndOfStream
    | anyio.EndOfStream
    | 
    | During handling of the above exception, another exception occurred:
    | 
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 179, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 491, in add_request_id
    |     response = await call_next(request)
    |                ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 155, in call_next
    |     raise RuntimeError("No response returned.")
    | RuntimeError: No response returned.
    +---------------- 2 ----------------
    | Traceback (most recent call last):
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/base.py", line 141, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/cors.py", line 85, in __call__
    |     await self.app(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 715, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 735, in app
    |     await route.handle(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 288, in handle
    |     await self.app(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 76, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/opt/app-root/lib64/python3.11/site-packages/starlette/routing.py", line 73, in app
    |     response = await f(request)
    |                ^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/routing.py", line 301, in app
    |     raw_response = await run_endpoint_function(
    |                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/fastapi/routing.py", line 212, in run_endpoint_function
    |     return await dependant.call(**values)
    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/api_server.py", line 348, in create_chat_completion
    |     generator = await handler.create_chat_completion(request, raw_request)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_chat.py", line 160, in create_chat_completion
    |     ) = await self._preprocess_chat(
    |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 496, in _preprocess_chat
    |     prompt_inputs = self._tokenize_prompt_input(
    |                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 326, in _tokenize_prompt_input
    |     return next(
    |            ^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 349, in _tokenize_prompt_inputs
    |     yield self._normalize_prompt_text_to_input(
    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/vllm/entrypoints/openai/serving_engine.py", line 234, in _normalize_prompt_text_to_input
    |     encoded = tokenizer(prompt, add_special_tokens=add_special_tokens)
    |               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2868, in __call__
    |     encodings = self._call_one(text=text, text_pair=text_pair, **all_kwargs)
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 2978, in _call_one
    |     return self.encode_plus(
    |            ^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_base.py", line 3054, in encode_plus
    |     return self._encode_plus(
    |            ^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 613, in _encode_plus
    |     batched_output = self._batch_encode_plus(
    |                      ^^^^^^^^^^^^^^^^^^^^^^^^
    |   File "/opt/app-root/lib64/python3.11/site-packages/transformers/tokenization_utils_fast.py", line 539, in _batch_encode_plus
    |     encodings = self._tokenizer.encode_batch(
    |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    | pyo3_runtime.PanicException: The global thread pool has not been initialized.: ThreadPoolBuildError { kind: GlobalPoolAlreadyInitialized }
    +------------------------------------
INFO:     127.0.0.1:60160 - "POST /v1/chat/completions HTTP/1.1" 500 Internal Server Error
INFO 01-31 03:41:02 metrics.py:449] Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.0%, CPU KV cache usage: 0.0%.