publications

2021

1. IJCNN

   Assisting the Adversary to Improve GAN Training

   Munk, Andreas, Harvey, William, and Wood, Frank

   In 2021 International Joint Conference on Neural Networks (IJCNN) Selected for oral presentation 2021

   Bib

   @inproceedings{munk2021assisting,
     title = {Assisting the Adversary to Improve GAN Training},
     author = {Munk, Andreas and Harvey, William and Wood, Frank},
     booktitle = {2021 International Joint Conference on Neural Networks (IJCNN)},
     pages = {1--8},
     year = {2021},
     note = {Selected for oral presentation},
     organization = {IEEE},
     abbr = {IJCNN},
     selected = {true}
   }
   

2020

1. Attention for Inference Compilation

   Harvey, William, Munk, Andreas, Baydin, Atılım Güneş, Bergholm, Alexander, and Wood, Frank

   Accepted as an extended abstract at the PROBPROG 2020 conference. 2020

   Bib

   @misc{ws-harvey2019attention,
     title = {Attention for Inference Compilation},
     author = {Harvey, William and Munk, Andreas and Baydin, At{\i}l{\i}m G{\"u}ne{\c{s}} and Bergholm, Alexander and Wood, Frank},
     note = {Accepted as an extended abstract at the PROBPROG 2020 conference.},
     year = {2020}
   }
   

2019

1. NeurIPS

   Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model

   Baydin, Atilim Gunes, Shao, Lei, Bhimji, Wahid, Heinrich, Lukas, Naderiparizi, Saeid, Munk, Andreas, Liu, Jialin, Gram-Hansen, Bradley, Louppe, Gilles, Meadows, Lawrence, Torr, Philip, Lee, Victor, Cranmer, Kyle, Prabhat, Mr., and Wood, Frank

   2019

   Bib

   @incollection{baydin2019efficient,
     title = {Efficient Probabilistic Inference in the Quest for Physics Beyond the Standard Model},
     booktitle = {Advances in Neural Information Processing Systems 32},
     author = {Baydin, Atilim Gunes and Shao, Lei and Bhimji, Wahid and Heinrich, Lukas and Naderiparizi, Saeid and Munk, Andreas and Liu, Jialin and {Gram-Hansen}, Bradley and Louppe, Gilles and Meadows, Lawrence and Torr, Philip and Lee, Victor and Cranmer, Kyle and Prabhat, Mr. and Wood, Frank},
     year = {2019},
     pages = {5459--5472},
     publisher = {Curran Associates, Inc.},
     copyright = {All rights reserved},
     selected = {true},
     abbr = {NeurIPS}
   }
   

2. Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale

   Baydin, Atilim Güneş, Shao, Lei, Bhimji, Wahid, Heinrich, Lukas, Meadows, Lawrence, Liu, Jialin, Munk, Andreas, Naderiparizi, Saeid, Gram-Hansen, Bradley, Louppe, Gilles, Ma, Mingfei, Zhao, Xiaohui, Torr, Philip, Lee, Victor, Cranmer, Kyle, Prabhat, , and Wood, Frank

   In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis 2019

   Bib Abs

   @inproceedings{baydin2019etalumis,
     title = {Etalumis: Bringing Probabilistic Programming to Scientific Simulators at Scale},
     shorttitle = {Etalumis},
     booktitle = {Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis},
     author = {Baydin, Atilim G{\"u}ne{\c s} and Shao, Lei and Bhimji, Wahid and Heinrich, Lukas and Meadows, Lawrence and Liu, Jialin and Munk, Andreas and Naderiparizi, Saeid and {Gram-Hansen}, Bradley and Louppe, Gilles and Ma, Mingfei and Zhao, Xiaohui and Torr, Philip and Lee, Victor and Cranmer, Kyle and Prabhat and Wood, Frank},
     year = {2019},
     month = nov,
     pages = {1--24},
     publisher = {Association for Computing Machinery},
     address = {New York, NY, USA},
     doi = {10.1145/3295500.3356180},
     copyright = {All rights reserved},
     isbn = {978-1-4503-6229-0},
     keywords = {deep learning,inference,probabilistic programming,simulation},
     series = {SC '19}
   }
   

   Probabilistic programming languages (PPLs) are receiving widespread attention for performing Bayesian inference in complex generative models. However, applications to science remain limited because of the impracticability of rewriting complex scientific simulators in a PPL, the computational cost of inference, and the lack of scalable implementations. To address these, we present a novel PPL framework that couples directly to existing scientific simulators through a cross-platform probabilistic execution protocol and provides Markov chain Monte Carlo (MCMC) and deep-learning-based inference compilation (IC) engines for tractable inference. To guide IC inference, we perform distributed training of a dynamic 3DCNN-LSTM architecture with a PyTorch-MPI-based framework on 1,024 32-core CPU nodes of the Cori supercomputer with a global mini-batch size of 128k: achieving a performance of 450 Tflop/s through enhancements to PyTorch. We demonstrate a Large Hadron Collider (LHC) use-case with the C++ Sherpa simulator and achieve the largest-scale posterior inference in a Turing-complete PPL.

3. Efficient Bayesian Inference for Nested Simulators

   Gram-Hansen, Bradley, Witt, Christian Schroeder, Zinkov, Robert, Naderiparizi, Saeid, Scibior, Adam, Munk, Andreas, Wood, Frank, Ghadiri, Mehrdad, Torr, Philip, Teh, Yee Whye, Baydin, Atilim Gunes, and Rainforth, Tom

   2019

   Bib Abs

   @article{gram-hansen2019efficient,
     title = {Efficient Bayesian Inference for Nested Simulators},
     author = {{Gram-Hansen}, Bradley and de Witt, Christian Schroeder and Zinkov, Robert and Naderiparizi, Saeid and Scibior, Adam and Munk, Andreas and Wood, Frank and Ghadiri, Mehrdad and Torr, Philip and Teh, Yee Whye and Baydin, Atilim Gunes and Rainforth, Tom},
     year = {2019},
     month = oct,
     copyright = {All rights reserved},
     language = {en}
   }
   

   We introduce two approaches for efficient and scalable inference in stochastic simulators for which the density cannot be evaluated directly due to, for example, rejection sampling loops.

4. Deep Probabilistic Surrogate Networks for Universal Simulator Approximation

   Munk, Andreas, Ścibior, Adam, Baydin, Atılım Güneş, Stewart, Andrew, Fernlund, Goran, Poursartip, Anoush, and Wood, Frank

   arXiv:1910.11950 [cs, stat] 2019

   Bib Abs

   @article{munk2019deep,
     title = {Deep Probabilistic Surrogate Networks for Universal Simulator Approximation},
     author = {Munk, Andreas and {\'S}cibior, Adam and Baydin, At{\i}l{\i}m G{\"u}ne{\c s} and Stewart, Andrew and Fernlund, Goran and Poursartip, Anoush and Wood, Frank},
     year = {2019},
     month = oct,
     archiveprefix = {arXiv},
     copyright = {All rights reserved},
     eprint = {1910.11950},
     eprinttype = {arxiv},
     journal = {arXiv:1910.11950 [cs, stat]},
     keywords = {Computer Science - Machine Learning,Statistics - Machine Learning},
     primaryclass = {cs, stat},
     selected = {true}
   }
   

   We present a framework for automatically structuring and training fast, approximate, deep neural surrogates of existing stochastic simulators. Unlike traditional approaches to surrogate modeling, our surrogates retain the interpretable structure of the reference simulators. The particular way we achieve this allows us to replace the reference simulator with the surrogate when undertaking amortized inference in the probabilistic programming sense. The fidelity and speed of our surrogates allow for not only faster "forward" stochastic simulation but also for accurate and substantially faster inference. We support these claims via experiments that involve a commercial composite-materials curing simulator. Employing our surrogate modeling technique makes inference an order of magnitude faster, opening up the possibility of doing simulator-based, non-invasive, just-in-time parts quality testing; in this case inferring safety-critical latent internal temperature profiles of composite materials undergoing curing from surface temperature profile measurements.

5. Amortized Rejection Sampling in Universal Probabilistic Programming

   Naderiparizi, Saeid, Ścibior, Adam, Munk, Andreas, Ghadiri, Mehrdad, Baydin, Atılım Güneş, Gram-Hansen, Bradley, de Witt, Christian Schroeder, Zinkov, Robert, Torr, Philip H. S., Rainforth, Tom, Teh, Yee Whye, and Wood, Frank

   arXiv:1910.09056 [cs, stat] 2019

   Bib Abs

   @article{naderiparizi2019amortized,
     title = {Amortized Rejection Sampling in Universal Probabilistic Programming},
     author = {Naderiparizi, Saeid and {\'S}cibior, Adam and Munk, Andreas and Ghadiri, Mehrdad and Baydin, At{\i}l{\i}m G{\"u}ne{\c s} and {Gram-Hansen}, Bradley and {de Witt}, Christian Schroeder and Zinkov, Robert and Torr, Philip H. S. and Rainforth, Tom and Teh, Yee Whye and Wood, Frank},
     year = {2019},
     month = nov,
     archiveprefix = {arXiv},
     copyright = {All rights reserved},
     eprint = {1910.09056},
     eprinttype = {arxiv},
     journal = {arXiv:1910.09056 [cs, stat]},
     keywords = {Computer Science - Artificial Intelligence,Computer Science - Machine Learning,Statistics - Machine Learning},
     primaryclass = {cs, stat}
   }
   

   Existing approaches to amortized inference in probabilistic programs with unbounded loops can produce estimators with infinite variance. An instance of this is importance sampling inference in programs that explicitly include rejection sampling as part of the user-programmed generative procedure. In this paper we develop a new and efficient amortized importance sampling estimator. We prove finite variance of our estimator and empirically demonstrate our method’s correctness and efficiency compared to existing alternatives on generative programs containing rejection sampling loops and discuss how to implement our method in a generic probabilistic programming framework.

2018

1. ICASSP

   Semi-Supervised Sleep-Stage Scoring Based on Single Channel EEG

   Munk, A. M., Olesen, K. V., Gangstad, S. W., and Hansen, L. K.

   In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) 2018

   Bib Abs

   @inproceedings{munk2018semisupervised,
     title = {Semi-Supervised Sleep-Stage Scoring Based on Single Channel EEG},
     booktitle = {2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
     author = {Munk, A. M. and Olesen, K. V. and Gangstad, S. W. and Hansen, L. K.},
     year = {2018},
     month = apr,
     pages = {2551--2555},
     issn = {2379-190X},
     doi = {10.1109/ICASSP.2018.8461982},
     copyright = {All rights reserved},
     keywords = {automatic sleep stage classification,Brain modeling,EEG,electroencephalography,Electroencephalography,fully-supervised model,Gaussian mixture model,generalizable Gaussian mixture model,learning (artificial intelligence),medical signal processing,non-negative matrix factorizaton,pattern classification,semi-supervised learning,semisupervised approach performs,semisupervised sleep-stage scoring,signal classification,sleep,Sleep,sleep stage scoring,sleep stages,Spectrogram,supervised classification algorithms,supervised machine learning algorithms,Training},
     abbr = {ICASSP},
     selected = {true}
   }
   

   The field of automatic sleep stage classification based on EEG has enjoyed substantial attention during the last decade, which has resulted in several supervised classification algorithms with highly encouraging performance. Such supervised machine learning algorithms require large training sets that have been manually labelled, and are time- and resource-consuming to acquire. Here we present a semi-supervised approach that can learn to distinguish the sleep stages from a one-night data set where only a fraction has been manually labelled. We show that for fractions larger than 50%, our semi-supervised approach performs as good as a similar, fully-supervised model.