JENT: Lag preditor health test
In the investigation of issue #21, it became clear that under certain circumstances (in that case, when the optimizer was turned on) long repeating sequences could come to dominate the entropy assessment of the source. In response to this observation, the optimizer was disabled (in commit 20184e97), which resolved this issue for the specific platforms where the problem was first reported. I think this type of failure might reoccur on other platforms and should be viewed as an observed failure mode. In terms of SP 800-90B, that suggests that the library should have some sort of health test that detects this failure mode. If we examine the entropy estimates produced by the SP 800-90B tests while the entropy source is in one of these failure modes, we find that the LRS, t-tuple, and Lag Predictor estimators showed unusually low assessments. The LRS and t-tuple estimators don't lend themselves to adaption to an online health test, but the Lag Predictor health test is amenable to this use. This patch provides an implementation of a variant of the Lag Predictor estimator (SP 800-90B Section 6.3.8) as a health test. It includes cutoffs targeted for the level of significance (alpha) used in the other health tests. Much like the Lag Prediction Estimator from SP 800-90B, this health test looks for both global predictability (successfully predicting more symbols than expected, given an assumed entropy level of `1/osr`) and local predictability (the presence of runs of successful predictions longer than would be expected, again given the assumed entropy level of `1/osr`). Either type of predictability triggers a health test failure. As a side observation, increasing the depth of the predictor (`JENT_LAG_HISTORY_SIZE`) seems to lead to worse (rather than better) predictions; if this test didn't perturb the noise source at all, this could not happen. It seems that this may instead be a consequence of increased churn in the RAM cache for larger history depth, so increasing the depth of the predictor has the side effect of making the system less predictable. Note: The lag predictor is currently disabled per default by not defining JENT_HEALTH_LAG_PREDICTOR. The patch is a first merge of the code into the current tree. Signed-off-by: Joshua E. Hill Signed-off-by:Stephan Mueller <smueller@chronox.de>
Loading
Please register or sign in to comment