In the following documentation: https://debezium.io/documentation/reference/connectors/postgresql.html

In the AWS RDS section it mentioned:

"
Use the RDS master account for replication as RDS currently does not support setting of REPLICATION privilege for another account.
"
 
It's true that you can't grant REPLICATION privilege to another account, but I managed to get this to work by:
 
1. Creating a new user and
2. Granting `rds_replication` to that user: 
 
 
grant rds_replication to my_user