我制作了一个实时的 AR 飞机识别器,以下是它背后的数学原理
2 分•作者: ananddhruv29•4 个月前
我一直在开发一款安卓应用,当你用手机指向天空时,它可以识别头顶的飞机。这款应用获取实时的 ADS-B 数据,并在相机画面上叠加飞机标签,但把数学计算弄对花了我比预想更长的时间,所以我写下了这篇文章。
这个问题听起来很简单,你有一个天空中的 GPS 坐标,以及你手中的 GPS 坐标。你需要一个像素。但在两者之间有四个不同的坐标空间,它们之间的转换具有隐式失效的符号约定,输出错误但没有错误提示。
流程如下:
```
大地坐标 (纬度, 经度, 高度)
↓ 平面地球近似 — 有效范围 <100 公里,在 50 海里范围内的误差 <2 像素
ENU — 东,北,上 (米)
↓ 来自安卓 TYPE_ROTATION_VECTOR 传感器的 R⊤
设备坐标系 (dX, dY, dZ)
↓ 一个符号翻转:Cz = −dZ
相机坐标系 (Cx, Cy, Cz)
↓ 透视除法 + FOV 归一化
屏幕像素 (Xpx, Ypx)
```
每个转换都难以理解的原因:
大地坐标 → ENU。东方向有一个余弦因子,大多数实现都忽略了它:E = Δλ × (π·RE/180) × cos(φ_user)。子午线向两极收敛,在纬度 25° 时,一度经度的米数比在赤道上少。如果没有它,东西方向的位置在赤道附近看起来是正确的,但随着纬度的增加,会悄悄地发散。
ENU → 设备坐标系。安卓的旋转矩阵 R 将设备轴映射到 ENU 世界轴。要反过来,你需要使用 R⊤。在安卓的行主序 FloatArray(9) 中,这意味着列索引,而不是行索引:
```
R (正向): dX = R[0]·E + R[1]·N + R[2]·U
R⊤ (逆向): dX = R[0]·E + R[3]·N + R[6]·U
```
这会产生完全不同的结果。两者都能编译通过,不会报错。
设备 → 相机坐标系。安卓的传感器定义 +Zd 指向屏幕外朝向你的脸。相机约定要求 +Cz 指向场景内。所以 Cz = −dZ,永远成立。这是纵向模式下唯一需要的修正。
相机 → 屏幕。在透视除法和 FOV 归一化之后,Y 轴翻转:Ypx = (1 − NDCy) × H/2。相机 +Cy 向上;屏幕 y=0 在顶部。如果我们错过了这一点,地平线上方的飞机将出现在屏幕中心下方。
实际捕获的值 (ATR72, 18,000 英尺):
```
用户:24.8600°N, 80.9813°E
飞机:24.9321°N, 81.0353°E
ENU:E=6,010 米 N=8,014 米 U=5,486 米
方位角 34.2° (北偏东),仰角 29.5°,距离 11.1 公里
相机坐标系 (在 R⊤ + 符号修正后):(729, 4692, 10077)
大小:11,140 米 ≈ 11,138 米 (ENU 距离)
屏幕 (1080×1997, θH=66°, θV=50°):(600 像素, 1 像素)
```
手机方位角 33.0°,飞机方位角 34.2° → 偏右 1.2°。
手机俯仰角 −4.3°,仰角 29.5° → 向上 33.8°,刚好在视锥体的上边缘内。物理上始终一致。
很乐意回答关于流程的任何阶段或任何其他感兴趣的问题。
查看原文
I've been building an Android app that identifies aircraft overhead when
you point your phone at the sky. The app fetches live ADS-B data and
overlays aircraft labels on the camera feed, but getting the math right
took much longer than I expected, so I wrote it all up.<p>The problem sounds simple, you have a GPS coordinate in the sky and a GPS
coordinate in your hand. You want a pixel. But there are four distinct
coordinate spaces between those two things, and the transitions between
them have sign conventions that fail silently, wrong output with no error.<p>The pipeline:<p><pre><code> Geodetic (lat, lon, alt)
↓ flat-earth approx — valid <100 km, error <2 px at 50 nm range
ENU — East, North, Up (metres)
↓ R⊤ from Android TYPE_ROTATION_VECTOR sensor
Device frame (dX, dY, dZ)
↓ one sign flip: Cz = −dZ
Camera frame (Cx, Cy, Cz)
↓ perspective divide + FOV normalisation
Screen pixels (Xpx, Ypx)
</code></pre>
Why each transition is non-obvious:<p>Geodetic → ENU. The East component has a cosine factor that most
implementations miss: E = Δλ × (π·RE/180) × cos(φ_user). Meridians
converge toward the poles, one degree of longitude is fewer metres at
latitude 25° than at the equator. Without it, East-West positions look
correct near the equator and quietly diverge as latitude increases.<p>ENU → Device frame. Android's rotation matrix R maps device axes to ENU
world axes. To go the other direction you use R⊤. In Android's row-major
FloatArray(9), this means column indices, not row indices:<p><pre><code> R (forward): dX = R[0]·E + R[1]·N + R[2]·U
R⊤ (inverse): dX = R[0]·E + R[3]·N + R[6]·U
</code></pre>
These produce completely different results. Both compile without complaint.<p>Device → Camera frame. Android's sensor defines +Zd as pointing out of
the screen toward your face. The camera convention requires +Cz to point
into the scene. So Cz = −dZ, always. This is the only correction needed
for portrait mode.<p>Camera → Screen. After the perspective divide and FOV normalisation, the
Y axis flips: Ypx = (1 − NDCy) × H/2. Camera +Cy is up; screen y=0 is
at the top. If we miss this, the aircraft above the horizon appears below screen
centre.<p>Real captured values (ATR72, 18,000 ft):<p><pre><code> User: 24.8600°N, 80.9813°E
Aircraft: 24.9321°N, 81.0353°E
ENU: E=6,010 m N=8,014 m U=5,486 m
Bearing 34.2° (NNE), Elevation 29.5°, Range 11.1 km
Camera frame (after R⊤ + sign fix): (729, 4692, 10077)
Magnitude: 11,140 m ≈ 11,138 m (ENU range)
Screen (1080×1997, θH=66°, θV=50°): (600 px, 1 px)
</code></pre>
Phone azimuth 33.0°, aircraft bearing 34.2° → 1.2° right of centre.
Phone pitched −4.3°, elevation 29.5° → net 33.8° up, just inside the
top edge of the frustum. Physically consistent throughout.<p>Happy to answer questions about any stage of the pipeline or about anything else, whatever is
interesting to anyone.