Some vllm operations (and dependencies) might be relying on torch.utils.cpp_extension._get_cuda_arch_flags (https://github.com/pytorch/pytorch/blob/v2.7.1/torch/utils/cpp_extension.py?plain=1#L2303-L2322) to infer what architectures to build JIT components for.

At build time it makes sense to build using TORCH_CUDA_ARCH_LIST to build for all of our intended target architectures, but at run time, it's better to allow _get_cuda_arch_flags choosing the correct flags automatically based on the detected GPUs, which is the default behaviour when TORCH_CUDA_ARCH_LIST is unset.