When I connect to some app through Fuse Console, there is Select a container dropdown menu in the left corner. There should be other deployed fuse containers listed and you can click on them and connect.

But the behaviour of this feature is very uncertain now : sometimes, the list is there, sometimes no. Sometimes you can connect to the app through the list, sometimes there is No Selected Container error - select-container.mkv

This feature worked on 7.4 on OCP 3, so it is regression. But I'm not considering it as a blocker. You can still go to Fuse console from some app and connect to containers from there.