1. 1. 1. 🐛 Describe the bug

Setting a `float64` tensor to [nn.RNN()](https://pytorch.org/docs/stable/generated/torch.nn.RNN.html) gets the error message as shown below:

```python
import torch
from torch import nn

my_tensor = torch.tensor([[8., -3., 5.]], dtype=torch.float64) # float64

torch.manual_seed(42)

rnn = nn.RNN(input_size=3, hidden_size=2)

rnn(input=my_tensor) # Error
```

> ValueError: input must have the type torch.float32, got type torch.float64

And, setting a `complex64` tensor to `nn.RNN()` gets the error message as shown below:

```python
import torch
from torch import nn

my_tensor = torch.tensor([[8.+0.j, -3.+0.j, 5.+0.j]]) # complex64

torch.manual_seed(42)

rnn = nn.RNN(input_size=3, hidden_size=2)

rnn(input=my_tensor) # Error
```

> ValueError: input must have the type torch.float32, got type torch.complex64

But setting a `float64` tensor or `complex64` tensor to `nn.RNN()` with `dtype=torch.float64` or `dtype=torch.complex64` respectively works as shown below:

```python
import torch
from torch import nn

my_tensor = torch.tensor([[8., -3., 5.]], dtype=torch.float64) # float64

torch.manual_seed(42)

1. ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
   rnn = nn.RNN(input_size=3, hidden_size=2, dtype=torch.float64)

rnn(input=my_tensor)

1. (tensor([[-1.0000, -0.9999]], dtype=torch.float64, grad_fn=<SqueezeBackward1>),
2. tensor([[-1.0000, -0.9999]], dtype=torch.float64, grad_fn=<SqueezeBackward1>))

my_tensor = torch.tensor([[8.+0.j, -3.+0.j, 5.+0.j]]) # complex64

torch.manual_seed(42)

1. ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
   rnn = nn.RNN(input_size=3, hidden_size=2, dtype=torch.complex64)

rnn(input=my_tensor)

1. (tensor([[ 0.9965-0.0006j, -0.9936-0.0332j]], grad_fn=<SqueezeBackward1>),
2. tensor([[ 0.9965-0.0006j, -0.9936-0.0332j]], grad_fn=<SqueezeBackward1>))
   ```

I think the error messages should be something like as shown below:

> ValueError: the dtype of the `input` tensor and `RNN()` must be the same but got `float64` and `float32` respectively

> ValueError: the dtype of the `input` tensor and `RNN()` must be the same but got `complex64` and `float32` respectively

1. 1. 1. Versions

```python
import torch

torch._version_ # '2.3.0'
```

cc @albanD @mruberry @jbschlosser @walterddr @mikaylagawarecki