Uploaded image for project: 'Debezium'
  1. Debezium
  2. DBZ-9182

Provide batching alternative to FieldToEmbedding SMT for LLM integrations

XMLWordPrintable

    • Icon: Feature Request Feature Request
    • Resolution: Unresolved
    • Icon: Major Major
    • 3.3-plan
    • None
    • core-library
    • None
    • False
    • Hide

      None

      Show
      None
    • False

      The `FieldToEmbedding` SMT, by the nature of being a single message transform, processes elements one-by-one, which can be detrimental to latency and throughput in particular when also supporting remote LLMs. Let's explore an alternative contract which allows to process records in batches, thus doing a single LLM invocation for calculating embeddings for an entire batch (provided of course a model supports this, but many do), and amortizing the cost of the call.

              Unassigned Unassigned
              gunnar.morling Gunnar Morling
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

                Created:
                Updated: