1. 1. 1. 🐛 Describe the bug

There is a consistency issue between CPU and CUDA implementations of `torch.nn.NLLLoss` when handling empty inputs with mismatched data types.

When provided with an empty `input` tensor (Float32) and a `weight` tensor of a different type (Float16):

• *CPU*: Correctly validates the types and raises `RuntimeError: expected scalar type Float but found Half`.
• *CUDA*: Skips the validation check, executes "successfully", and returns `tensor(nan, device='cuda:0')`.

This violates device parity; code that works on CUDA (due to the skipped check) will crash when moved to CPU.

Here is a [gist](https://colab.research.google.com/gist/jiren-the-gray/cee0fb151e7bd53765939d46eac420b3/nllloss_bug.ipynb)

*Minimal reproduction*

```python
import torch

print(f"PyTorch Version:

")
print(f"CUDA Available:

")

1. Setup: Empty tensors with MISMATCHED types
2. Input is Float32, Weight is Float16 (Half)
   input_t = torch.tensor([], dtype=torch.float32).reshape(0, 0)
   weight_t = torch.tensor([], dtype=torch.float16)
   target_t = torch.tensor([], dtype=torch.long)

1. 1. Test CPU (Correct Behavior)
   print("\n--- Testing CPU ---")
   try:
   criterion = torch.nn.NLLLoss(weight=weight_t)
   res = criterion(input_t, target_t)
   print(f"CPU Result: {res}")
   except RuntimeError as e:
   print(f"CPU caught expected error: {e}")
   
   # 2. Test CUDA (Buggy Behavior)
   if torch.cuda.is_available():
   print("\n--- Testing CUDA ---")
   try:
   criterion_gpu = torch.nn.NLLLoss(weight=weight_t.cuda())
   res = criterion_gpu(input_t.cuda(), target_t.cuda())
   
   print(f"CUDA Result: {res}

   ")
   print(">> Bug: CUDA skipped the type check!")
   except RuntimeError as e:
   print(f"CUDA caught expected error:

   {e}

   ")
   ```

*Expected behavior*
CUDA should match the CPU behavior and raise a `RuntimeError` indicating that the `weight` tensor dtype (Half) does not match the `input` tensor dtype (Float).

*Actual behavior*
```
— Testing CPU —
CPU caught expected error: expected scalar type Float but found Half

— Testing CUDA —
CUDA Result: tensor(nan, device='cuda:0')
>> Bug: CUDA skipped the type check!
```

*Additional context*
This appears to happen because the CUDA kernel (or the dispatcher) checks for input size 0 and exits early before validating the weight tensor compatibility.

1. 1. 1. Versions

Collecting environment information...
PyTorch version: 2.9.0+cu126
Is debug build: False
CUDA used to build PyTorch: 12.6
ROCM used to build PyTorch: N/A

OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04.2) 11.4.0
Clang version: Could not collect
CMake version: version 3.31.10
Libc version: glibc-2.35

Python version: 3.12.12 (main, Oct 10 2025, 08:52:57) [GCC 11.4.0] (64-bit runtime)
Python platform: Linux-6.6.105+-x86_64-with-glibc2.35
Is CUDA available: True
CUDA runtime version: 12.5.82
CUDA_MODULE_LOADING set to:
GPU models and configuration: GPU 0: Tesla T4
Nvidia driver version: 550.54.15
cuDNN version: Probably one of the following:
/usr/lib/x86_64-linux-gnu/libcudnn.so.9.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_adv.so.9.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_cnn.so.9.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_engines_precompiled.so.9.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_engines_runtime_compiled.so.9.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_graph.so.9.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_heuristic.so.9.2.1
/usr/lib/x86_64-linux-gnu/libcudnn_ops.so.9.2.1
Is XPU available: False
HIP runtime version: N/A
MIOpen runtime version: N/A
Is XNNPACK available: True

CPU:
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 46 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 2
On-line CPU(s) list: 0,1
Vendor ID: GenuineIntel
Model name: Intel(R) Xeon(R) CPU @ 2.30GHz
CPU family: 6
Model: 63
Thread(s) per core: 2
Core(s) per socket: 1
Socket(s): 1
Stepping: 0
BogoMIPS: 4599.99
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm abm ssbd ibrs ibpb stibp fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat md_clear arch_capabilities
Hypervisor vendor: KVM
Virtualization type: full
L1d cache: 32 KiB (1 instance)
L1i cache: 32 KiB (1 instance)
L2 cache: 256 KiB (1 instance)
L3 cache: 45 MiB (1 instance)
NUMA node(s): 1
NUMA node0 CPU(s): 0,1
Vulnerability Gather data sampling: Not affected
Vulnerability Indirect target selection: Vulnerable
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Mitigation; PTE Inversion
Vulnerability Mds: Vulnerable; SMT Host state unknown
Vulnerability Meltdown: Vulnerable
Vulnerability Mmio stale data: Vulnerable
Vulnerability Reg file data sampling: Not affected
Vulnerability Retbleed: Vulnerable
Vulnerability Spec rstack overflow: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers
Vulnerability Spectre v2: Vulnerable; IBPB: disabled; STIBP: disabled; PBRSB-eIBRS: Not affected; BHI: Vulnerable
Vulnerability Srbds: Not affected
Vulnerability Tsa: Not affected
Vulnerability Tsx async abort: Not affected

Versions of relevant libraries:
[pip3] intel-cmplr-lib-ur==2025.3.1
[pip3] intel-openmp==2025.3.1
[pip3] mkl==2025.3.0
[pip3] numpy==2.0.2
[pip3] nvidia-cublas-cu12==12.6.4.1
[pip3] nvidia-cuda-cupti-cu12==12.6.80
[pip3] nvidia-cuda-nvrtc-cu12==12.6.77
[pip3] nvidia-cuda-runtime-cu12==12.6.77
[pip3] nvidia-cudnn-cu12==9.10.2.21
[pip3] nvidia-cufft-cu12==11.3.0.4
[pip3] nvidia-curand-cu12==10.3.7.77
[pip3] nvidia-cusolver-cu12==11.7.1.2
[pip3] nvidia-cusparse-cu12==12.5.4.2
[pip3] nvidia-cusparselt-cu12==0.7.1
[pip3] nvidia-nccl-cu12==2.27.5
[pip3] nvidia-nvjitlink-cu12==12.6.85
[pip3] nvidia-nvtx-cu12==12.6.77
[pip3] nvtx==0.2.14
[pip3] onemkl-license==2025.3.0
[pip3] optree==0.18.0
[pip3] tbb==2022.3.0
[pip3] tcmlib==1.4.1
[pip3] torch==2.9.0+cu126
[pip3] torchao==0.10.0
[pip3] torchaudio==2.9.0+cu126
[pip3] torchdata==0.11.0
[pip3] torchsummary==1.5.1
[pip3] torchtune==0.6.1
[pip3] torchvision==0.24.0+cu126
[pip3] triton==3.5.0
[pip3] umf==1.0.2
[conda] Could not collect

cc @ptrblck @msaroufim @eqy @jerryzh168 @tinglvv @nWEIdia @malfet