A Pre-Validation Study of Legacy Data

In 2013-2014, Dr. Balch briefly worked on a project as a contractor at AFRL Wright-Patterson. The purpose of this project was to develop methods adequate for the validation of reduced-order models used to simulate the performance of structures in hypersonic flight. As part of this project, Dr. Balch explored the possibility of using aerothermal data obtained from NASA Langley's 8' High-Temperature Tunnel in 1986. 

At first glance, the legacy data in technical report NASA-TP-2631 would seem ideal for testing piston theory, one of the reduced-order physics models that the Wright-Patterson team wanted to use in their work. However, there were unexpected phenomena in the data. Specifically, the pressure ratio across the oblique shock between the freestream flow and the experimental apparatus was roughly 5% off of what one would expect it to be. More troublingly, the pressure distribution across the domes in the experimental apparatus were unexpectedly asymmetric. Dr. Balch devised a method that hybridized Bayesian inference and frequentist significance testing in order to determine whether or not the oddities in the data were the result of random experimental error. They were not. Moreover, it was not possible to determine which of the competing physical explanations for this bias was correct. Three explanations were explored, but all three were plausible given the data at hand. Lacking a conclusive explanation and no way of uncovering it by experimenting with the original set up, it was thus not possible to correct for this relatively large bias. 

Ultimately, this meant that the legacy aerothermal data were not suitable for validating reduced-order models. That finding was a disappointment for the team, to be sure, but it was an important finding nonetheless. And indeed, this is a fairly typical finding. Legacy data which were considered groundbreaking in their time (and rightly so!) are often not up to the rigors of present-day validation work. 

Dr. Balch presented a paper on this analysis in 2014.