Sensor Recorder Pro turns an iPhone into a low-cost, reproducible, multi-sensor data recorder for VIO, SLAM, robotics, XR, embodied AI, Physical AI, and field experiments.
- App Store: Sensor Recorder Pro
- Open data: Baidu Netdisk, password:
inv0
The app records synchronized real-world signals from iPhone hardware:
- Up to three selected camera streams from wide, ultra-wide, telephoto, and front cameras. Unsupported MultiCam combinations are automatically trimmed.
- Per-frame camera metadata: timestamp, exposure, ISO, resolution, and intrinsics.
- Audio from
audio.m4a. - Raw accelerometer and gyroscope data.
- Gyro-keyed IMU rows.
- CoreMotion device motion, including attitude, gravity, user acceleration, and rotation rate.
- Magnetometer, barometer, and Geo location.
- LiDAR raw depth maps on supported devices, saved as 16-bit PNG depth in millimeters with a CSV index.
- Adaptive preview layout for 1-3 RGB cameras plus optional depth preview.
- A
meta.jsonmanifest with device metadata, capture settings, schemas, codecs, and timestamp semantics.
Each session is saved as a folder named SR_yyyy-MM-dd_HH-mm-ss/. The phone keeps recording simple and robust: videos stay as MP4, audio stays as M4A, sensor streams stay as CSV, and offline tools convert the session for visualization and analysis.
LiDAR depth output, when enabled, is stored as raw frame files:
lidar_depth/
├── depth_000001.png
├── depth_000002.png
└── ...
lidar_depth_info.csv
Each .png is a 16-bit grayscale depth map in millimeters. Pixel value 0 means invalid depth, and value / 1000 gives meters. When LiDAR Depth is enabled, camera capture and recording are capped at 10Hz; high resolutions may still drop frames. lidar_depth_info.csv stores frame time, UTC time, file name, resolution, pixel format, depth range, scale, and intrinsics.
Every stream carries two timestamps when available:
sensor_sec: monotonic sensor time for camera, IMU, audio, motion, and sensor fusion alignment.utc_sec: Unix UTC time for wall-clock correlation, Geo data, external logs, and experiment notes.
Use sensor_sec for sensor alignment. Use utc_sec when correlating with the outside world.
- Open
SensorRecorder.xcodeprojin Xcode. - Set your signing team in
Project -> Signing & Capabilities. - Connect an iPhone that supports MultiCam capture.
- Build and run on device.
The current capture pipeline targets iOS 15.4+.
Convert a completed session to Rerun:
python3 -m pip install rerun-sdk numpy pillow
python3 tools/convert_recording.py /path/to/SR_yyyy-MM-dd_HH-mm-ss -o recording.rrd
rerun recording.rrdThe converter uses wide_info.csv, ultra_info.csv, optional tele_info.csv, and optional front_info.csv as the source of truth for camera frame time. It decodes MP4 frames with local ffmpeg, logs images into Rerun, and restores every logged frame onto the recorded sensor_time timeline from sensor_sec. It also logs utc_time when available.
By default video is written to Rerun at up to 5fps to keep long recordings manageable. Use --video-fps 0 to write every frame.
LiDAR depth PNGs are logged into Rerun as depth images, and every depth frame is reconstructed into a point cloud from fx/fy/cx/cy in lidar_depth_info.csv. Use --depth-pixel-stride 1 for full-resolution point clouds; the default stride is 2 to keep .rrd files manageable.
The saved Rerun layout shows:
- Top: ultra-wide, wide, optional telephoto, and optional front image streams.
- Lower left: IMU acceleration, gyro, and raw
audio.m4awaveform. - Lower right: attitude roll/pitch/yaw and Geo ENU curves in meters.
Sensor Recorder Pro 把 iPhone 变成一个低成本、可复现、多模态的真实世界数据采集器，面向 VIO、SLAM、机器人、XR、具身智能、Physical AI 和科学实验。
- App Store 下载：Sensor Recorder Pro
- 开放数据：百度网盘，密码：
inv0
这个 App 可以同步记录 iPhone 硬件中的多源传感器数据：
- 从 wide、ultra-wide、telephoto、front 中任选最多三路相机视频。不支持的 MultiCam 组合会自动裁剪。
- 每帧相机信息：时间戳、曝光、ISO、分辨率、相机内参。
audio.m4a音频。- 原始加速度计和陀螺仪。
- gyro 对齐的 IMU 数据。
- CoreMotion device motion，包括姿态、重力、用户加速度和旋转速度。
- 磁力计、气压计和 Geo 位置。
- 支持 LiDAR 的设备会保存原始 depth map，以 16-bit PNG 毫米深度图加 CSV 索引保存。
- 预览区会根据 1-3 路 RGB 相机和可选 Depth tile 自动铺满横屏。
meta.json，记录设备信息、采集设置、schema、codec 和时间模型。
每次录制会保存为一个 SR_yyyy-MM-dd_HH-mm-ss/ 文件夹。手机端只负责稳定记录原始数据：视频保存为 MP4，音频保存为 M4A，传感器保存为 CSV，后处理工具再把 session 转换成适合可视化和分析的格式。
LiDAR depth 开启时会固定输出：
lidar_depth/
├── depth_000001.png
├── depth_000002.png
└── ...
lidar_depth_info.csv
每个 .png 是 16-bit 灰度深度图，单位是毫米。像素值 0 表示无效深度，value / 1000 得到米制深度。开启 LiDAR Depth 后，相机采集和录制帧率上限为 10Hz；高分辨率下仍可能丢帧。lidar_depth_info.csv 保存帧时间、UTC 时间、文件名、分辨率、像素格式、深度范围、scale 和内参。
每条数据尽量保留两种时间：
sensor_sec：单调递增的传感器时间，用于相机、IMU、音频、姿态等多源数据对齐。utc_sec：Unix UTC 时间，用于和真实世界时间、Geo、外部日志、实验记录关联。
传感器融合和对齐优先使用 sensor_sec。需要和外部世界关联时使用 utc_sec。
- 用 Xcode 打开
SensorRecorder.xcodeproj。 - 在
Project -> Signing & Capabilities设置自己的签名团队。 - 连接支持 MultiCam 的 iPhone。
- 在真机上编译运行。
当前采集链路目标版本是 iOS 15.4+。
把一次录制转换成 Rerun：
python3 -m pip install rerun-sdk numpy pillow
python3 tools/convert_recording.py /path/to/SR_yyyy-MM-dd_HH-mm-ss -o recording.rrd
rerun recording.rrd转换器以 wide_info.csv、ultra_info.csv、可选的 tele_info.csv 和可选的 front_info.csv 作为相机帧时间戳的真值来源。它用本地 ffmpeg 解码 MP4，把图像写入 Rerun，并把每一帧恢复到原始 sensor_sec 对应的 sensor_time 时间轴；可用时也会写入 utc_time。
默认视频最多按 5fps 写入 Rerun，避免长时间录制生成过大的 .rrd 文件。使用 --video-fps 0 可以写入每一帧。
LiDAR depth PNG 会作为 depth image 写入 Rerun，并根据 lidar_depth_info.csv 里的 fx/fy/cx/cy 为每一帧恢复点云。默认点云像素 stride 为 2，避免 .rrd 文件过大；使用 --depth-pixel-stride 1 可输出全分辨率点云。
Rerun 默认布局包括：
- 上方：ultra-wide、wide、可选 telephoto 和可选 front 图像。
- 左下：IMU acceleration、gyro、从
audio.m4a解码的原始音频波形。 - 右下：attitude roll/pitch/yaw 和 Geo ENU 米制曲线。