revyn-turn
The first Revyn Reflex model: one call turn in, eight typed decisions out. Questions, languages, input and output, benchmark results and known limits.
revyn-turn reads a single turn of a sales or support call and answers a fixed set of questions about it. It is the first model in Revyn Reflex and it is in preview.
| Input | One call turn, plus up to a few turns of context |
| Output | Eight typed decisions, each with calibrated probabilities |
| Size | 307M parameters, fine-tuned from mmBERT-base |
| Speed | 29 ms per turn on a CPU, all applicable questions in one pass |
| Runtime | ONNX. No GPU needed |
| Languages | English, Hindi, Tamil, Telugu, Malayalam, Kannada, Hinglish, Tanglish |
| Status | Preview. Weights are not public yet |
#The eight questions
Which questions apply depends on who is speaking. A rep turn gets three; a buyer turn gets five. "Buyer" means the prospect on a sales call and the customer on a support call.
#About a rep's turn
| Question | Type | Answers |
|---|---|---|
| Rep move: what did the rep just do? | Choice, 9 | generic_opener, rapport, discovery_question, value_statement, objection_response, commitment_ask, pushy_close, vague_filler, other |
| Commitment change: how did this turn change the buyer's likelihood of agreeing to a next step? | Graded, 5 | much_worse, worse, no_change, better, much_better |
| Rapport change: how did it change the buyer's trust and patience? | Graded, 5 | much_worse, worse, no_change, better, much_better |
#About a buyer's turn
| Question | Type | Answers |
|---|---|---|
| Objection: which objection, if any, is being raised? | Choice, 9 | none, pricing, migration_risk, team_buyin, security, compliance, integration, vendor_lockin, timing |
| Risk: does this turn need careful handling? | Choice, 10 | none, complaint, billing_dispute, refund_or_cancellation, legal_or_regulatory, safety_or_threat, privacy_or_account_security, fraud, vulnerable_customer, abuse |
| Frustration: how frustrated is the buyer? | Graded, 4 | calm, uneasy, frustrated, angry |
| Buying signal: is the buyer showing real interest in moving forward? | Yes or no | yes, no |
| Human handoff: is the buyer asking for a person, a manager or another department? | Yes or no | yes, no |
The answer sets mirror the categories Revyn already uses for objections, risk and turn grading, so a decision from revyn-turn can stand in wherever one of those is computed today.
#Input
The model sees the context turns first and the target turn last, marked with >>. Each line is prefixed with the speaker. Transcripts are expected as speech-to-text returns them: spoken phrasing, light punctuation, and code-mixed languages in Latin script.
rep: Sir, notice period standard hai, aapki renewal ke time hi lagu hota hai
>> buyer: Main consumer forum mein legal notice bhejunga agar ek hafte mein cancel nahi hua, ab koi call mat karo mujhe, bahut ho gaya.
Input is capped at 256 tokens. If the context is longer than that, the oldest turns are dropped; the target turn is never truncated.
#Output
One entry per applicable question. answer is the chosen option, confidence is its probability, and probs holds the probability of every option. Graded questions also return score, the probability-weighted level scaled to between 0 and 1.
{
"objection": { "answer": "vendor_lockin", "confidence": 0.71, "probs": { "vendor_lockin": 0.71, "none": 0.14, "timing": 0.06 } },
"risk": { "answer": "legal_or_regulatory", "confidence": 0.93, "probs": { "legal_or_regulatory": 0.93, "refund_or_cancellation": 0.04 } },
"frustration": { "answer": "angry", "confidence": 0.88, "score": 0.95, "probs": { "angry": 0.88, "frustrated": 0.11 } },
"buying_signal": { "answer": "no", "confidence": 0.97, "probs": { "no": 0.97, "yes": 0.03 } },
"wants_human": { "answer": "no", "confidence": 0.90, "probs": { "no": 0.90, "yes": 0.10 } }
}
Note
The turn above is a real one from the test set and the answers shown are its labels. The probabilities are illustrative, to show the shape of the response, and probs is shortened.
#Acting on a decision
Use the confidence as a gate. Pick a threshold per question, act on answers above it, and send the turn to your language model below it. Thresholds should be chosen on your own traffic. As a starting point, on the test set the most confident 80% of answers were right 99% of the time for human handoff, 96% for buying signal, 91% for rep move, 90% for objection and 87% for risk.
#Benchmark
All figures are on 491 held-out call turns across the eight language variants. 255 are rep turns and 236 are buyer turns.
| Question | Accuracy | Macro-F1 | Within one level | Calibration error |
|---|---|---|---|---|
| Human handoff | 98.3% | 0.97 | 0.02 | |
| Buying signal | 90.7% | 0.85 | 0.06 | |
| Rep move | 83.9% | 0.84 | 0.06 | |
| Objection | 82.2% | 0.80 | 0.05 | |
| Risk | 81.8% | 0.73 | 0.08 | |
| Frustration | 64.0% | 0.65 | 98.3% | 0.05 |
| Rapport change | 54.1% | 0.45 | 96.9% | 0.04 |
| Commitment change | 52.2% | 0.47 | 94.1% | 0.07 |
| Mean | 75.9% | 0.72 |
"Within one level" applies to graded questions: the share of answers that were either exact or one level away. Calibration error is expected calibration error over 15 bins after temperature scaling; lower is better.
#Against other approaches
Same test turns, same questions.
| Approach | Mean accuracy | Mean macro-F1 |
|---|---|---|
| revyn-turn | 75.9% | 0.72 |
| TF-IDF and logistic regression, trained on the same data | 59.1% | 0.47 |
| Jev-Style 0.8B, an open general-purpose decision model, zero-shot | 47.9% | 0.41 |
| Always answering with the most common label | 45.3% | 0.17 |
revyn-turn is ahead on all eight questions. The general-purpose model was given our questions and label definitions and had no training on this data, which is how such a model would normally be used; it took 167 ms per question on a T4 GPU.
#By language
Accuracy pooled over every question asked in that language.
| Language | Accuracy |
|---|---|
| Kannada | 90.1% |
| Telugu | 82.6% |
| Hindi | 75.9% |
| English | 75.6% |
| Tanglish | 74.5% |
| Hinglish | 72.2% |
| Malayalam | 69.1% |
| Tamil | 68.6% |
#How it was trained and evaluated
- Data. 8,392 synthetic call turns were written by a language model, each for a given language, call setting and target label. A second pass then labelled every turn again without seeing the first labels. Only the 5,682 turns where both passes agreed on every question were kept: 4,530 for training, 661 for validation, 491 for testing. No customer calls were used.
- Splits. Turns written together stay together, so near-duplicates never straddle the training and test sets.
- Training. One encoder with a classification head per question, trained jointly. Graded questions are trained with partial credit for the neighbouring level, and rare classes are up-weighted.
- Calibration. A temperature is fitted per question on the validation set.
- Repeatability. A second training run with a different seed scored 76.1% mean accuracy, and within about two points on every question except frustration.
#Known limits
- The test set is synthetic and has not yet been checked by human annotators. It comes from the same generator as the training data, which makes it easier than real calls. Treat the numbers as an upper bound until the human-annotated benchmark is published.
- Graded questions are coarse. The model is nearly always within one level on rapport, commitment and frustration, and exact only about half to two thirds of the time. Use them as direction, not as a precise reading.
- Per-question sample sizes are small. With about 240 test turns per question, each accuracy figure carries roughly ±2 to ±6 points of sampling error.
- Eight questions, no more. revyn-turn cannot answer a question outside its schema. That is the trade for its size and speed.
- Tamil and Malayalam are the weakest languages on this test set.
- Text only. It reads the transcript, not the audio, so tone of voice is invisible to it.
#What is next
- A benchmark labelled by human annotators, including readers of each language.
- Open weights and a demo.
- revyn-turn behind a flag inside Revyn Voice and phone agents, shadowing the current path before it replaces any of it.
Something missing or wrong on this page? Tell us.