CSI-TextBench: A Dataset and Benchmark for Language-Grounded Ambient Sensing Perception
Abstract
Indoor context deciphered by ambient sensing underpins many applications, from assisted living to smart buildings, yet remains weakly connected to the language-aligned paradigm of modern multimodal systems. Through billions of IoT devices, ambient WiFi Channel State Information (CSI) captures the indoor context of human activity and environmental dynamics passively via ubiquitous, always-on infrastructure. However, unlike vision or audio, CSI lacks naturally paired text like captions or transcripts, limiting its integration with language-driven AI systems. We present CSI-TextBench, a large-scale dataset and benchmark for aligning CSI data with natural language, unifying 3 sensing benchmarks into 285,790 samples across 7 tasks and pairing them with 2,291 hierarchically structured, label-derived descriptions via many-to-many assignment to construct 2.0 million CSI-text training tuples. Benchmarking 5 alignment methods from 4 paradigm families, we show that contrastive CSI-text alignment matches supervised classifiers without class labels, adapts to unseen classes with few samples, and responds to open-vocabulary natural-language prompts. A single unified model supports compositional scene understanding, answering joint queries over activity, proximity, and identity, achieving 86.9% top-1 accuracy on 140 composite scenes, outperforming task-specific classifier ensembles. CSI-TextBench enables ubiquitous ambient sensing as a language-indexed perception layer, enabling indoor contextual awareness in language-driven AI systems.
The following analysis is generated by AI and may contain errors. It is separate from the authors’ abstract and is not peer review.
Research summary
CSI-TextBench introduces a large-scale dataset and benchmark for aligning WiFi CSI with natural language, demonstrating that contrastive learning can enable zero-shot and compositional scene understanding in ambient sensing. The paper makes a significant contribution by unifying heterogeneous sensing benchmarks into a language-compatible format, providing rigorous evaluation protocols for open-vocabulary and few-shot settings, and showing that a single unified model can effectively handle multi-attribute scene queries, thereby establishing a foundation for language-grounded ambient intelligence.
Methodology
The paper proposes a systematic framework for aligning WiFi Channel State Information (CSI) with natural language. The core methodological contribution is not a new architecture but a rigorous benchmarking protocol and dataset construction pipeline. The authors unify three heterogeneous CSI datasets (CSI-Bench, XRF55, SignFi) into a standardized tensor format and generate text descriptions via hand-written templates rather than LLMs, ensuring label grounding. They evaluate five alignment objectives (CLIP-InfoNCE, SigLIP, LanguageBind, Barlow Twins, Masked Prediction) within a shared dual-encoder architecture (1D ResNet-18 for CSI, frozen CLIP ViT-B/32 for text). The methodology is sound, particularly in its handling of the "many-to-many" correspondence between signals and text, which is a unique challenge in sensing modalities lacking native captions. The use of frozen text encoders isolates the learning of the CSI representation, providing a clean experimental design.
Results & evaluation
The experimental evaluation is comprehensive, covering zero-shot classification, few-shot transfer, open-vocabulary prompting, and compositional scene understanding. The results demonstrate that contrastive alignment (specifically CLIP-InfoNCE) can match supervised classifiers without class labels, a significant finding for ambient sensing. The "Layer 3" experiments on compositional scene understanding (86.9% accuracy on joint queries) are particularly strong, showing that a single unified model can outperform task-specific ensembles. The ablations on text granularity and encoder choice are well-executed. However, the reliance on template-based text generation limits the semantic richness compared to true natural language, and the performance drop on fine-grained unseen classes (11.3% zero-shot) highlights the limits of current alignment techniques for highly specific gestures.
Reproducibility
The paper provides high reproducibility. The authors release the dataset, code, and benchmark. The preprocessing pipeline is detailed, including specific normalization and subcarrier alignment steps. The training hyperparameters (learning rate, batch size, optimizer) are explicitly stated. The use of standard, publicly available base models (CLIP, ResNet) further aids reproducibility. The distinction between Level 1 and Level 2 captions is clearly defined, allowing other researchers to replicate the specific evaluation protocols.
Limitations
The primary limitation is the synthetic nature of the text data. Since descriptions are derived from labels via templates, the model may not generalize to truly free-form, noisy, or context-rich natural language that does not strictly adhere to the template structures. The paper acknowledges this by testing "human-written free-form" language, but the gap between template-based and truly open-ended language remains. Additionally, the cross-environment and cross-device transfer performance is noted to be constrained by the inherent domain sensitivity of WiFi CSI, which is a fundamental physical limitation rather than a model flaw, but it limits the immediate practical deployment of the aligned model in unseen environments.
Broader impact
This work bridges the gap between passive sensing and language-driven AI, enabling ambient intelligence systems to be queried in natural language. This has significant implications for assistive living, smart buildings, and privacy-preserving monitoring (as CSI does not capture visual or audio content). By providing a standardized benchmark, it facilitates future research into RF-language alignment, potentially leading to more intuitive and flexible human-computer interfaces for indoor environments. The approach is generalizable to other sensing modalities like radar or mmWave, offering a template for language-grounded perception in non-visual domains. CSI-TextBench introduces a large-scale dataset and benchmark for aligning WiFi CSI with natural language, demonstrating that contrastive learning can enable zero-shot and compositional scene understanding in ambient sensing. The paper makes a significant contribution by unifying heterogeneous sensing benchmarks into a language-compatible format, providing rigorous evaluation protocols for open-vocabulary and few-shot settings, and showing that a single unified model can effectively handle multi-attribute scene queries, thereby establishing a foundation for language-grounded ambient intelligence.
Author of this paper? Send a correction.