As part of the EBS to OrgID transition, the ingress service is now supporting anemic tenants. We realize that not all downstream apps may be capable of handling anemic tenants right now and have created a transition plan so that ingress can move forward without impacting these apps. In addition, it gives us an opportunity to streamline the way we send out upload notifications.

It’s important to note that all incoming upload messages, including anemic tenants, will be sent to `platform.upload.available`, while only messages with account numbers will be sent to the legacy custom topics. (ex. platform.upload.<app_name>)

Currently, each onboarded service gets an individual kafka topic that they consume from. This is problematic because we have to maintain a mapping of content types to topics, we have to create a producer for each topic, and we have some topics that are created but never used.

In this “single topic” method, we’ll instead send all messages to one topic, platform.upload.available. Downstream apps are expected to begin consuming from this topic and use kafka headers to filter on the messages they actually want to process. 

We’ve had Puptoo using this method for months with no noticeable performance hit over this time. By filtering on the kafka headers, the message is not opened and the JSON does not have to be parsed, making it a good method for consuming only messages you care about.

We’d like to see every app that consumes messages from ingress to begin using the single topic method by the end of August, but ideally as soon as your service is able to consume anemic tenant events.

For more info, see our migration proposal doc[1] or contact us on slack at #proj-orgid-migration. 

 [1]https://docs.google.com/document/d/1-ngQq-MydgXICMnO7aTeKaQRMsYqCG4Uw42lSwSbBdw/edit