TUPLE-Bench: Evaluating and Mitigating Privacy-Utility Trade-offs in Typed Decision Models
Provides novel benchmarking methodology and mitigation techniques for privacy-utility tradeoffs in AI decision systems
AI Summary
Researchers introduce TUPLE-Bench to evaluate privacy leakage in typed decision models, finding that 91.88%-99.93% of responses reveal sensitive information through output fields.
Excerpt
Typed decision models often return more than a selected category. They also expose probabilities for each candidate and, sometimes, a separate confidence score. These fields support downstream decisions but may reveal information beyond a recipient's authorization. We introduce TUPLE-Bench (Typed decision models' Utility and Privacy Leakage Evaluation Benchmark), a benchmark of paired synthetic cases in which recipients may learn a coarse operational category but not its fine-grained cause. Acro