Vol. 1 No. 4 (2026)

Articles

Heavy-Tailed Q-Exponential Priors for Robust Deep Uncertainty Propagation Under Shift

Published 2026-09-16

  • Peng Yao
    • ,
  • Dorothy Yu

Peng Yao
Department of Computer Science and Engineering, Faculty of Engineering, Chinese University of Hong Kong, Hong Kong, Hong Kong SAR, China

Dorothy Yu
Department of Computer Science, School of Computing and Decision Sciences, Hang Seng University of Hong Kong, Hong Kong, Hong Kong SAR, China

Data Science and Computational Intelligence, Volume 1, Issue 4, 2026 cover

Abstract

The deployment of deep neural networks in safety-critical domains necessitates highly reliable uncertainty quantification, particularly when models encounter data that deviates from the training distribution. Traditional Bayesian neural networks frequently rely on standard Gaussian priors, which possess exponentially decaying tails. These thin-tailed priors tend to be overly restrictive, leading to unwarranted overconfidence and poor uncertainty propagation under covariate shift or out-of-distribution scenarios. To address these limitations, this paper proposes the utilization of heavy-tailed Q-exponential priors derived from non-extensive Tsallis statistics. By parameterizing the tail behavior through a structural index, the Q-exponential family offers a flexible and robust mechanism for regularizing deep networks. The heavy tails allow for greater weight variability, which in turn preserves epistemic uncertainty when propagating activations through multiple non-linear layers under domain shift. This study systematically formulates the theoretical foundations of Q-exponential priors, integrates them into a variational inference framework, and evaluates their performance across complex image classification benchmarks subjected to diverse synthetic corruptions. Extensive empirical analysis demonstrates that the proposed approach significantly improves calibration metrics and predictive log-likelihoods on shifted data compared to conventional Gaussian and Laplace priors. The findings underscore the critical role of prior tail behavior in deep uncertainty quantification and establish Q-exponential distributions as a highly effective tool for building resilient machine learning systems in unpredictable environments.

Keywords

Bayesian Neural Networks, Uncertainty Quantification, Domain Shift, Heavy-Tailed Distributions, Tsallis Statistics

PDF

Author Biography

Peng Yao

Editorial correspondence on behalf of the authors

References

  1. Wang, J., Xu, Y., Li, Y., Khargonekar, P., & Faruque, M. A. A. (2026). CRUISE: Vision-Language Model-Guided Uncertainty-Aware Cross-Modal Sensor Fusion for Robust Autonomous Driving. arXiv preprint arXiv:2608.09202.
  2. Qu, W., Shao, Y., Meng, L., Huang, X., & Xiao, L. (2024, June). A conditional denoising diffusion probabilistic model for point cloud upsampling. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 20786-20795). IEEE.
  3. Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., et al. (2016). Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA, 316(22), 2402–2410.
  4. Li, Q., Luo, T., & Liao, J. (2025). Theory-inspired deep multi-view multi-label learning with incomplete views and noisy labels. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 20706-20715).
  5. Zhang, J., & Tang, H. (2025). Resolving Task Objective Conflicts in Unified Model via Task-Aware Mixture-of-Experts. arXiv preprint arXiv:2506.03591.
  6. Sang, Y. (2026, March). Adaptive Quantization Strategies for Robust ML Inference Under Distribution Shift. In Proceedings of the 2026 5th International Conference on Cyber Security, Artificial Intelligence and Digital Economy (pp. 362-368).
  7. Hu, H., Yang, Y., Zhao, Q., & Zhang, C. (2022, June). On the role of discount factor in offline reinforcement learning. In International conference on machine learning (pp. 9072-9098). PMLR.
  8. Yu, Z., Dai, L., Xu, S., Gao, S., & Ho, C. P. (2023). Fast bellman updates for wasserstein distributionally robust mdps. Advances in Neural Information Processing Systems, 36, 30554-30578.
  9. Zhang, Z., Yin, Z., Wen, J., Sun, L., Su, S., & Yu, P. S. (2021). DeepBlue: Bi-layered LSTM for tweet popularity estimation. IEEE Transactions on Knowledge and Data Engineering, 34(10), 4737-4752.
  10. Acquah, C., Hoehn, S., Krul, S., Jockusch, S., Yang, S., Seth, S. K., ... & Crespo-Hernández, C. E. (2024). Electronic relaxation pathways in thio-acridone and thio-coumarin: two heavy-atom-free photosensitizers absorbing visible light. Physical Chemistry Chemical Physics, 26(46), 28980-28991.
  11. Yao, Z., Hawi, P., Aitharaju, V., Mahishi, J., & Ghanem, R. (2026). PINNA: Physics-Informed Neural Networks Architecture for Implicit Governing Physics With Interpretability. International Journal for Numerical Methods in Engineering, 127(17), e70399.
  12. Zhang, J., Luo, H., Yang, H., Xu, X., Wu, Q., Shi, Y., ... & Wang, J. (2023). Neuraldome: A neural modeling pipeline on multi-view human-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 8834-8845).
  13. Chai, L.; Zhang, J.; Li, H.; Cui, C.; Jiang, J.; Zheng, B.; Wu, L.; Jiang, L. Investigation of thermomorphogenesis-related genes for a multi-silique trait in Brassica napus by comparative transcriptome analysis. Front. Genet. 2021, 12, 678804.
  14. Chang, Z., Obite, C., Zhou, S., & Lan, S. (2024). Deep Q-Exponential Processes. arXiv preprint arXiv:2410.22119.
  15. Tian, Y. (2026, January). A Data-Driven Energy Efficiency Improvement and Cost Optimisation Model for New Energy Electric Vehicles. In 2026 IEEE International Conference on Power, Electronics and Green Energy (ICPEGE) (pp. 304-308). IEEE.
  16. López-Acosta, N. P. (2019, November). Evaluating rigidity index, OCR, and su from dilatometer data in soft to firm clays. In Geotechnical Engineering in the XXI Century: Lessons learned and future challenges: Proceedings of the XVI Pan-American Conference on Soil Mechanics and Geotechnical Engineering (XVI PCSMGE), 17-20 November 2019, Cancun, Mexico (p. 286). IOS Press.
  17. He, H., Sun, Z., Fan, J., & Mok, P. Y. (2025). TiPGAN: High-quality tileable textures synthesis with intrinsic priors for cloth digitization applications. Computer-Aided Design, 183, 103866.
  18. Gao, S., & Shi, L. (2015). Selecting the best simulated design with the expected opportunity cost bound. IEEE Transactions on Automatic Control, 60(10), 2785-2790.
  19. Zhao, J., Zhang, C., Qin, M., & Yang, P. (2025). QuantFactor REINFORCE: mining steady formulaic alpha factors with variance-bounded REINFORCE. IEEE Transactions on Signal Processing, 73, 2448-2463.
  20. Zhang, F., Gao, S., & Song, J. (2024, August). Offline Contextual Learning with Correlated Bayesian Beliefs. In 2024 IEEE 20th International Conference on Automation Science and Engineering (CASE) (pp. 2496-2500). IEEE.
  21. Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017). Dermatologist-level classification of skin cancer with deep neural networks. Nature, 542(7639), 115–118.
  22. M. Emani et al., ‘‘A comprehensive evaluation of novel AI accelerators for deep learning workloads,’’ in Proc. IEEE/ACM Int. Workshop Perform. Modeling, Benchmarking Simulation High Perform. Comput. Syst. (PMBS), Nov. 2022, pp. 13–25.
  23. Obite, C. P., Chang, Z., Wu, K., & Lan, S. (2025, May). Bayesian regularization of latent representation. In International Conference on Learning Representations (Vol. 2025, pp. 12805-12823).
  24. Li, Y. (2026, June). AI-augmented automated triage for system error reporting using log and memory dump analysis. In 2026 7th International Conference on Big Data & Artificial Intelligence & Software Engineering (ICBASE) (pp. 572-577). IEEE.
  25. Chen, L., Wang, J., Mortlock, T., Khargonekar, P., & Al Faruque, M. A. (2025, June). Hyperdimensional uncertainty quantification for multimodal uncertainty fusion in autonomous vehicles perception. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 22306-22316). IEEE.
  26. Lyu, S., Mao, S., & Ma, Y. (2025). Unified Long Video Inpainting and Outpainting via Overlapping High-Order Co-Denoising. arXiv preprint arXiv:2511.03272.
  27. Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., et al. (2018). Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell, 172(5), 1122–1131.
  28. Wu, J., Chen, S., Heo, I., Gutfraind, S., Liu, S., Li, C., ... & Sharps, M. (2025). Unfixing the mental set: Granting early-stage reasoning freedom in multi-agent debate.
  29. Xia, F., Li, B., An, B., Zachman, M. J., Xie, X., Liu, Y., ... & Cheng, Y. (2024). Cooperative Atomically Dispersed Fe–N4 and Sn–N x Moieties for Durable and More Active Oxygen Electroreduction in Fuel Cells. Journal of the American Chemical Society, 146(49), 33569-33578.
  30. Sha, L., Wang, Y., Meng, P., Deng, Y., Chen, T., Zhang, X., ... & Xu, Q. (2025). Pharmacological inhibition of PSPH reduces serine levels and epileptic seizures. Nature Chemical Biology, 21(11), 1742-1753.
  31. Liu, Y., Cai, Q., Wang, C., Yang, J., Fan, H., Dong, J., & Chen, S. (2024). Geometry-enhanced attentive multi-view stereo for challenging matching scenarios. IEEE Transactions on Circuits and Systems for Video Technology, 34(8), 7401-7416.