我制作了一个实时的 AR 飞机识别器,以下是它背后的数学原理

2作者: ananddhruv294 个月前
我一直在开发一款安卓应用,当你用手机指向天空时,它可以识别头顶的飞机。这款应用获取实时的 ADS-B 数据,并在相机画面上叠加飞机标签,但把数学计算弄对花了我比预想更长的时间,所以我写下了这篇文章。 这个问题听起来很简单,你有一个天空中的 GPS 坐标,以及你手中的 GPS 坐标。你需要一个像素。但在两者之间有四个不同的坐标空间,它们之间的转换具有隐式失效的符号约定,输出错误但没有错误提示。 流程如下: ``` 大地坐标 (纬度, 经度, 高度) ↓ 平面地球近似 — 有效范围 <100 公里,在 50 海里范围内的误差 <2 像素 ENU — 东,北,上 (米) ↓ 来自安卓 TYPE_ROTATION_VECTOR 传感器的 R⊤ 设备坐标系 (dX, dY, dZ) ↓ 一个符号翻转:Cz = −dZ 相机坐标系 (Cx, Cy, Cz) ↓ 透视除法 + FOV 归一化 屏幕像素 (Xpx, Ypx) ``` 每个转换都难以理解的原因: 大地坐标 → ENU。东方向有一个余弦因子,大多数实现都忽略了它:E = Δλ × (π·RE/180) × cos(φ_user)。子午线向两极收敛,在纬度 25° 时,一度经度的米数比在赤道上少。如果没有它,东西方向的位置在赤道附近看起来是正确的,但随着纬度的增加,会悄悄地发散。 ENU → 设备坐标系。安卓的旋转矩阵 R 将设备轴映射到 ENU 世界轴。要反过来,你需要使用 R⊤。在安卓的行主序 FloatArray(9) 中,这意味着列索引,而不是行索引: ``` R (正向): dX = R[0]·E + R[1]·N + R[2]·U R⊤ (逆向): dX = R[0]·E + R[3]·N + R[6]·U ``` 这会产生完全不同的结果。两者都能编译通过,不会报错。 设备 → 相机坐标系。安卓的传感器定义 +Zd 指向屏幕外朝向你的脸。相机约定要求 +Cz 指向场景内。所以 Cz = −dZ,永远成立。这是纵向模式下唯一需要的修正。 相机 → 屏幕。在透视除法和 FOV 归一化之后,Y 轴翻转:Ypx = (1 − NDCy) × H/2。相机 +Cy 向上;屏幕 y=0 在顶部。如果我们错过了这一点,地平线上方的飞机将出现在屏幕中心下方。 实际捕获的值 (ATR72, 18,000 英尺): ``` 用户:24.8600°N, 80.9813°E 飞机:24.9321°N, 81.0353°E ENU:E=6,010 米 N=8,014 米 U=5,486 米 方位角 34.2° (北偏东),仰角 29.5°,距离 11.1 公里 相机坐标系 (在 R⊤ + 符号修正后):(729, 4692, 10077) 大小:11,140 米 ≈ 11,138 米 (ENU 距离) 屏幕 (1080×1997, θH=66°, θV=50°):(600 像素, 1 像素) ``` 手机方位角 33.0°,飞机方位角 34.2° → 偏右 1.2°。 手机俯仰角 −4.3°,仰角 29.5° → 向上 33.8°,刚好在视锥体的上边缘内。物理上始终一致。 很乐意回答关于流程的任何阶段或任何其他感兴趣的问题。
查看原文
I&#x27;ve been building an Android app that identifies aircraft overhead when you point your phone at the sky. The app fetches live ADS-B data and overlays aircraft labels on the camera feed, but getting the math right took much longer than I expected, so I wrote it all up.<p>The problem sounds simple, you have a GPS coordinate in the sky and a GPS coordinate in your hand. You want a pixel. But there are four distinct coordinate spaces between those two things, and the transitions between them have sign conventions that fail silently, wrong output with no error.<p>The pipeline:<p><pre><code> Geodetic (lat, lon, alt) ↓ flat-earth approx — valid &lt;100 km, error &lt;2 px at 50 nm range ENU — East, North, Up (metres) ↓ R⊤ from Android TYPE_ROTATION_VECTOR sensor Device frame (dX, dY, dZ) ↓ one sign flip: Cz = −dZ Camera frame (Cx, Cy, Cz) ↓ perspective divide + FOV normalisation Screen pixels (Xpx, Ypx) </code></pre> Why each transition is non-obvious:<p>Geodetic → ENU. The East component has a cosine factor that most implementations miss: E = Δλ × (π·RE&#x2F;180) × cos(φ_user). Meridians converge toward the poles, one degree of longitude is fewer metres at latitude 25° than at the equator. Without it, East-West positions look correct near the equator and quietly diverge as latitude increases.<p>ENU → Device frame. Android&#x27;s rotation matrix R maps device axes to ENU world axes. To go the other direction you use R⊤. In Android&#x27;s row-major FloatArray(9), this means column indices, not row indices:<p><pre><code> R (forward): dX = R[0]·E + R[1]·N + R[2]·U R⊤ (inverse): dX = R[0]·E + R[3]·N + R[6]·U </code></pre> These produce completely different results. Both compile without complaint.<p>Device → Camera frame. Android&#x27;s sensor defines +Zd as pointing out of the screen toward your face. The camera convention requires +Cz to point into the scene. So Cz = −dZ, always. This is the only correction needed for portrait mode.<p>Camera → Screen. After the perspective divide and FOV normalisation, the Y axis flips: Ypx = (1 − NDCy) × H&#x2F;2. Camera +Cy is up; screen y=0 is at the top. If we miss this, the aircraft above the horizon appears below screen centre.<p>Real captured values (ATR72, 18,000 ft):<p><pre><code> User: 24.8600°N, 80.9813°E Aircraft: 24.9321°N, 81.0353°E ENU: E=6,010 m N=8,014 m U=5,486 m Bearing 34.2° (NNE), Elevation 29.5°, Range 11.1 km Camera frame (after R⊤ + sign fix): (729, 4692, 10077) Magnitude: 11,140 m ≈ 11,138 m (ENU range) Screen (1080×1997, θH=66°, θV=50°): (600 px, 1 px) </code></pre> Phone azimuth 33.0°, aircraft bearing 34.2° → 1.2° right of centre. Phone pitched −4.3°, elevation 29.5° → net 33.8° up, just inside the top edge of the frustum. Physically consistent throughout.<p>Happy to answer questions about any stage of the pipeline or about anything else, whatever is interesting to anyone.