revyn-turn

The first Revyn Reflex model: one call turn in, eight typed decisions out. Questions, languages, input and output, benchmark results and known limits.

revyn-turn reads a single turn of a sales or support call and answers a fixed set of questions about it. It is the first model in Revyn Reflex and it is in preview.

InputOne call turn, plus up to a few turns of context
OutputEight typed decisions, each with calibrated probabilities
Size307M parameters, fine-tuned from mmBERT-base
Speed29 ms per turn on a CPU, all applicable questions in one pass
RuntimeONNX. No GPU needed
LanguagesEnglish, Hindi, Tamil, Telugu, Malayalam, Kannada, Hinglish, Tanglish
StatusPreview. Weights are not public yet

#The eight questions

Which questions apply depends on who is speaking. A rep turn gets three; a buyer turn gets five. "Buyer" means the prospect on a sales call and the customer on a support call.

#About a rep's turn

QuestionTypeAnswers
Rep move: what did the rep just do?Choice, 9generic_opener, rapport, discovery_question, value_statement, objection_response, commitment_ask, pushy_close, vague_filler, other
Commitment change: how did this turn change the buyer's likelihood of agreeing to a next step?Graded, 5much_worse, worse, no_change, better, much_better
Rapport change: how did it change the buyer's trust and patience?Graded, 5much_worse, worse, no_change, better, much_better

#About a buyer's turn

QuestionTypeAnswers
Objection: which objection, if any, is being raised?Choice, 9none, pricing, migration_risk, team_buyin, security, compliance, integration, vendor_lockin, timing
Risk: does this turn need careful handling?Choice, 10none, complaint, billing_dispute, refund_or_cancellation, legal_or_regulatory, safety_or_threat, privacy_or_account_security, fraud, vulnerable_customer, abuse
Frustration: how frustrated is the buyer?Graded, 4calm, uneasy, frustrated, angry
Buying signal: is the buyer showing real interest in moving forward?Yes or noyes, no
Human handoff: is the buyer asking for a person, a manager or another department?Yes or noyes, no

The answer sets mirror the categories Revyn already uses for objections, risk and turn grading, so a decision from revyn-turn can stand in wherever one of those is computed today.

#Input

The model sees the context turns first and the target turn last, marked with >>. Each line is prefixed with the speaker. Transcripts are expected as speech-to-text returns them: spoken phrasing, light punctuation, and code-mixed languages in Latin script.

text
rep: Sir, notice period standard hai, aapki renewal ke time hi lagu hota hai
>> buyer: Main consumer forum mein legal notice bhejunga agar ek hafte mein cancel nahi hua, ab koi call mat karo mujhe, bahut ho gaya.

Input is capped at 256 tokens. If the context is longer than that, the oldest turns are dropped; the target turn is never truncated.

#Output

One entry per applicable question. answer is the chosen option, confidence is its probability, and probs holds the probability of every option. Graded questions also return score, the probability-weighted level scaled to between 0 and 1.

json
{
  "objection":     { "answer": "vendor_lockin", "confidence": 0.71, "probs": { "vendor_lockin": 0.71, "none": 0.14, "timing": 0.06 } },
  "risk":          { "answer": "legal_or_regulatory", "confidence": 0.93, "probs": { "legal_or_regulatory": 0.93, "refund_or_cancellation": 0.04 } },
  "frustration":   { "answer": "angry", "confidence": 0.88, "score": 0.95, "probs": { "angry": 0.88, "frustrated": 0.11 } },
  "buying_signal": { "answer": "no", "confidence": 0.97, "probs": { "no": 0.97, "yes": 0.03 } },
  "wants_human":   { "answer": "no", "confidence": 0.90, "probs": { "no": 0.90, "yes": 0.10 } }
}

Note

The turn above is a real one from the test set and the answers shown are its labels. The probabilities are illustrative, to show the shape of the response, and probs is shortened.

#Acting on a decision

Use the confidence as a gate. Pick a threshold per question, act on answers above it, and send the turn to your language model below it. Thresholds should be chosen on your own traffic. As a starting point, on the test set the most confident 80% of answers were right 99% of the time for human handoff, 96% for buying signal, 91% for rep move, 90% for objection and 87% for risk.

#Benchmark

All figures are on 491 held-out call turns across the eight language variants. 255 are rep turns and 236 are buyer turns.

QuestionAccuracyMacro-F1Within one levelCalibration error
Human handoff98.3%0.970.02
Buying signal90.7%0.850.06
Rep move83.9%0.840.06
Objection82.2%0.800.05
Risk81.8%0.730.08
Frustration64.0%0.6598.3%0.05
Rapport change54.1%0.4596.9%0.04
Commitment change52.2%0.4794.1%0.07
Mean75.9%0.72

"Within one level" applies to graded questions: the share of answers that were either exact or one level away. Calibration error is expected calibration error over 15 bins after temperature scaling; lower is better.

#Against other approaches

Same test turns, same questions.

ApproachMean accuracyMean macro-F1
revyn-turn75.9%0.72
TF-IDF and logistic regression, trained on the same data59.1%0.47
Jev-Style 0.8B, an open general-purpose decision model, zero-shot47.9%0.41
Always answering with the most common label45.3%0.17

revyn-turn is ahead on all eight questions. The general-purpose model was given our questions and label definitions and had no training on this data, which is how such a model would normally be used; it took 167 ms per question on a T4 GPU.

#By language

Accuracy pooled over every question asked in that language.

LanguageAccuracy
Kannada90.1%
Telugu82.6%
Hindi75.9%
English75.6%
Tanglish74.5%
Hinglish72.2%
Malayalam69.1%
Tamil68.6%

#How it was trained and evaluated

  • Data. 8,392 synthetic call turns were written by a language model, each for a given language, call setting and target label. A second pass then labelled every turn again without seeing the first labels. Only the 5,682 turns where both passes agreed on every question were kept: 4,530 for training, 661 for validation, 491 for testing. No customer calls were used.
  • Splits. Turns written together stay together, so near-duplicates never straddle the training and test sets.
  • Training. One encoder with a classification head per question, trained jointly. Graded questions are trained with partial credit for the neighbouring level, and rare classes are up-weighted.
  • Calibration. A temperature is fitted per question on the validation set.
  • Repeatability. A second training run with a different seed scored 76.1% mean accuracy, and within about two points on every question except frustration.

#Known limits

  • The test set is synthetic and has not yet been checked by human annotators. It comes from the same generator as the training data, which makes it easier than real calls. Treat the numbers as an upper bound until the human-annotated benchmark is published.
  • Graded questions are coarse. The model is nearly always within one level on rapport, commitment and frustration, and exact only about half to two thirds of the time. Use them as direction, not as a precise reading.
  • Per-question sample sizes are small. With about 240 test turns per question, each accuracy figure carries roughly ±2 to ±6 points of sampling error.
  • Eight questions, no more. revyn-turn cannot answer a question outside its schema. That is the trade for its size and speed.
  • Tamil and Malayalam are the weakest languages on this test set.
  • Text only. It reads the transcript, not the audio, so tone of voice is invisible to it.

#What is next

  1. A benchmark labelled by human annotators, including readers of each language.
  2. Open weights and a demo.
  3. revyn-turn behind a flag inside Revyn Voice and phone agents, shadowing the current path before it replaces any of it.

Something missing or wrong on this page? Tell us.