-
Feature Request
-
Resolution: Unresolved
-
Minor
-
None
-
2.14.0 GA
-
None
-
False
-
-
False
-
Not Started
-
Not Started
-
Not Started
-
Not Started
-
Not Started
-
Not Started
-
-
When working with some APIs like the ones for LLMs, "standard" rates are like 1-5 $/million tokens.
We now have a custom policy (or maybe already builtin in newer versions) to count the tokens emitted in a particular request. However, we can not add any cost linked to those metrics because it seems that unit costs are limited to 4 decimals (0.0001 $/metric), whereas we would need at least 6 for 1$/million token.
Ideally, we should be able to go down to at least 8 decimals (in case of 1 cent/million token), or even no specific lower limits (except the one on the parameter type)