In 2015, a colleague working as a satellite navigator came to Dr. Balch with a simple complaint, "My collision risk numbers don't make any sense." Dr. Balch looked into how collision risk was being calculated in conjunction analysis; that is, the analysis that a satellite navigator does when his satellite is projected to pass near another satellite or piece of debris. Dr. Balch quickly came to the conclusion that his colleague was right: The "probability" of collision numbers didn't make any sense. The problem was a phenomenon called probability dilution. As uncertainty in the satellite trajectory estimate increased, the risk of collision appeared to decrease. And the idea that noisier data make a system safer is foolish on its face. In fact, under ordinary operating conditions, probability dilution would lead an unwary satellite navigator to under-predict their collision risk exposure by several orders of magnitude; that is, if they took epistemic "probability" of collision at face value, which thankfully, some satellite navigators do not.
In 2016, Dr. Balch presented a paper detailing his early work on the problem. Since then, we have developed a proprietary algorithm for computing collision risk in rigorous frequentist terms. This algorithm delivers a collision risk metric that is statistically reliable in the same way that the ellipse (or ellipsoid) overlap approach is, while also being flexible in the same way that "probability" of collision was supposed to be. Our algorithm is also more efficient than ellipse overlap, in terms of its ability to discriminate between conjunctions that will or will not result in a collision. Moreover, this work led to a break-through in our general understanding of epistemic uncertainties resulting from statistical inference. Specifically, we found that representing such uncertainties probabilistically results in a phenomenon that can be aptly described as false confidence. Probability dilution in satellite conjunction analysis is a manifestation of this more general defect. Dr. Balch has written a paper with co-authors Ryan Martin and Scott Ferson detailing these findings, Satellite Conjunction Analysis and the False Confidence Theorem, which was published in the July 2019 issue of Proceedings of the Royal Society A: Mathematical, Physical, and Engineering Sciences.R-Scripts for producing the Figures One, Two, and Three from "Satellite Conjunction Analysis and the False Confidence Theorem" are attached below. |

PROJECTS >