-
Feature Request
-
Resolution: Unresolved
-
Major
-
None
-
None
-
False
-
-
False
The `FieldToEmbedding` SMT, by the nature of being a single message transform, processes elements one-by-one, which can be detrimental to latency and throughput in particular when also supporting remote LLMs. Let's explore an alternative contract which allows to process records in batches, thus doing a single LLM invocation for calculating embeddings for an entire batch (provided of course a model supports this, but many do), and amortizing the cost of the call.