Your documentation at:

https://debezium.io/documentation/reference/connectors/postgresql.html

Says:

The wal2json plug-in does not support special values, such as NaN or infinity, for floating point types.

But the wal2json plugin at:

https://github.com/eulerto/wal2json/blob/master/wal2json.c

Appears to serialize NaN etc as a string "NaN"

Furthermore, your Wal2JsonColumnValue.java converts doubles using Double.valueOf() which does support NaN etc.

So it might work already, and it looks like it could easily work if it does not.

Sorry for not testing - I am not actually using your product, but was just studying the source code in order to understand the underlying PG replication protocols for an unrelated project - and am just sending this in case it is of interest.