Question - can we get the token numbers input and output from the query transaction

[converseKarl]

use for logging/tracking/thresholds and bill costings

[adhityan]

Today, you can get that from the Langsmith integration. It's unclear to me what other tools are out there that people might be interested in using to track this. Suggestions?

[converseKarl]

Not everyone wants langsmith. It lacks in many areas but I can write my own in out token tracker with threshold alerting and map to analytics for billing.

[converseKarl]

You might want to check as openai for example provides input tokens, completion and total. I'm sure its now capable of returning token usage, completion and total, meaning its easier to track the cost. I would say this is a seriously key feature to have in the library crucial to track cost otherwise it becomes a serious problem. Please can yo bump this as a high priority? and amazon bedrock.

https://js.langchain.com/v0.1/docs/modules/model_io/chat/token_usage_tracking/

[adhityan]

Token used is now sent back in every query response. Check out the updated documentation at https://llm-tools.mintlify.app/ for more details.