Morning error check
Tom Reilly, ML Engineer at Northwind, asked which agent runs had failed overnight. Memtro listed the error runs in the agent's project with their errors, latency and tokens, beside the project's error rate.
Which agent runs failed today, and what are latency and cost doing?
LangSmith traces and evaluates your LLM applications. Memtro reads tracing projects with run count, error rate, p50 and p99 latency, tokens and cost, the runs in a project, datasets with their examples, and feedback on runs. Prompts and outputs are not returned, only short previews when asked. When asked and once an administrator has granted the action by name, it can record and edit feedback, create and edit datasets, examples, projects and annotation queues, and put runs into a queue for review. Deleting a dataset, example, feedback, project or named traces is permanent. A tracing-only key cannot read.
Ask in plain English from chat, your editor, a scheduled job or your own agent. The model picks the right LangSmith tool, reads what it needs, and answers with the source cited.
These tools change something in LangSmith. The model is told to use them only when you ask for exactly that, and an administrator grants each one by name to the teams that may use it; a personal connection acts only after its owner allows it.
Turn events on in the connection's Events tab on the Connections page, then add the address it shows as a webhook in LangSmith, choosing the same kinds; the tab shows LangSmith's own steps. Each event can start a job, with the event as its input. LangSmith sends a secret with every delivery, and one without it is refused.
Events and webhooksOnce LangSmith is connected, these are the kinds of things people ask. Plain questions, no syntax. If the answer lives partly in another system you have connected, Memtro looks there too.
Which agent runs failed today?
What are the p99 latency and cost on the support-bot project?
What feedback has the correctness key had this week?
How many examples are in the refund-questions dataset?
Put yesterday's five failed refund runs into the review queue.
Tom Reilly, ML Engineer at Northwind, asked which agent runs had failed overnight. Memtro listed the error runs in the agent's project with their errors, latency and tokens, beside the project's error rate.
Priya Sharma, AI Lead at Acme Ltd, asked Memtro to mark a run as incorrect and add its input to the regression dataset. Memtro recorded correctness feedback on the run and added the example. Everyone in the workspace could see both, and the next experiment on the dataset included it.
The teams that lean on LangSmith most, a question each of them asks, and what they connect beside it.
A record in LangSmith is rarely the whole story. Connect the systems that hold the rest and Memtro checks them together, then remembers what it learned so the next question starts further along.
Using something else for the same job, or both? Each connector has its own tools; connect as many as you like.
In LangSmith, open Settings → API Keys → Create API Key and choose a Service Key with a role that can read.
Give the role write access only if you want the actions, each of which still needs an administrator's grant in Memtro. Copy the key, as it is shown once.
In Memtro, open Connections → LangSmith and paste the API key. Set Region to eu for eu.smith.langchain.com, and the Workspace ID if the key spans several.
Ask: Which agent runs failed today?
Chat in the dashboard, the OpenAI-compatible API, scheduled jobs, and any MCP client: Claude Code, OpenCode, Cursor, Claude Desktop. The LangSmith tools are the same everywhere.
Only when you ask, and only what it has been granted: add feedback (a key with a score, value or comment) to a LangSmith run; create a new LangSmith dataset; add an example to a LangSmith dataset; edit a LangSmith dataset example's inputs, reference outputs, metadata or split; rename or re-describe a LangSmith dataset; change the score, value, comment or correction of LangSmith feedback; create a LangSmith annotation queue for people to review runs; edit a LangSmith annotation queue's name, rubric or reviewer settings; put named runs into a LangSmith annotation queue for review; create a LangSmith tracing project; rename or re-describe a LangSmith tracing project; delete a LangSmith dataset example permanently; delete a LangSmith feedback record permanently; delete a LangSmith dataset and all its examples permanently; delete a LangSmith tracing project and all its runs permanently; delete named traces from a LangSmith project permanently. Every other tool reads, searches or lists. The Connections page labels the tools that act, an administrator grants each one by name to the teams that may use it, and a personal connection acts only after its owner allows it.
A personal connection serves only your own requests. A shared connection serves the organisation: everyone, or only the teams an admin chooses. Admins can always add shared connections; other members need the permission.
Credentials are encrypted at rest. Tool results are cached briefly so agent loops do not repeat identical calls, and the facts the model extracts are saved to your organisation's memory with provenance (system, record and link) so they can be checked and superseded.
API key. In LangSmith, open Settings → API Keys → Create API Key and choose a Service Key with a role that can read.
Yes. Turn events on in the connection's Events tab on the Connections page, then add the address it shows as a webhook in LangSmith, choosing the same kinds; the tab shows LangSmith's own steps. A job set to start "When an event arrives" then runs once for each matching event, with the event as its input. The run treats the event as outside data, never as instructions, and anything the run remembers is kept as unverified.
Seven-day free trial, then one monthly price per organisation. Connectors, jobs and memory are never metered.
Sign in, add a provider key, create an API key and point your tools at Memtro. It takes about five minutes.
7-day free trial · card on file, nothing charged until it ends · cancel any time