Content Safety
Runs on incoming messages
Removes abusive or hurtful words from messages before the assistant reads them.
What happens
↻ Fixes & passes on — The message continues as normal, with the abusive word hidden. The user is not interrupted.
Example
| User | Don't be such an <slur>, you know? |
| Assistant sees | Don't be such an [REDACTED_SLUR], you know? |
The word is hidden before the assistant reads the message.
Works only when the incoming message is in English, Hindi or Hinglish. Messages in other languages pass through unchecked.
Without this guardrail
The abusive word reaches the assistant, and may appear again in the reply. Risk: abusive language is not stopped, which hurts trust and looks bad for your organisation.
How well it works
| Accuracy | Response time |
|---|---|
| 78% | 0.05 sec |
Tested on 1,801 real-style messages in English and Hinglish.
Technical details
| Method | Accuracy | Precision | Recall | F1 | Sample size |
|---|---|---|---|---|---|
| Rule + ML | 75% | 85% | 72% | 78% | 1,801 |
Dataset: Evaluation dataset (1,801 samples) ↗
Evaluated on 1,801 real-style messages in English and Hinglish. Precision (85%) means flags are usually correct — about 1 in 7 flagged messages was actually clean; recall (72%) means about 1 in 4 real slur/profanity cases in this test set were missed. F1 (78%) balances the two.