Structured Reverse Mode Automatic Differentiation in Nested Monte Carlo Simulations

Abstract

In many practical large scale computational problems, the calculation of partial derivatives of the object function f with respect to input parameters are entailed and the dimension of inputs n is much larger the one of outputs m. The use of reverse mode automatic differentiation (AD) is mostly efficient as it computes the gradient in the same amount of runtime as f regardless of the input dimension n. However, it demands excessive memory. To enjoy the runtime efficiency of reverse mode without paying unaffordable memory, structured reverse mode has been proposed and succeeded in several applications. Due to the fundamental difficulty in automatic structure detection, structured reverse mode has not been fully automated. This thesis, instead of trying to solve to structure detection problem for a completely generic piece of code, is devoted to the analysis and implementation of deploying structured reverse mode to a generic class of problems with a known structure, nested Monte Carlo simulations. We reveal the general structure pattern of Monte Carlo simulations in financial applications. Space/time tradeoff on deploying structured reverse mode are discussed in details and numerical experiments using Variable Annuity program are conducted to corroborate the analysis. Significant memory and runtime reductions are observed. We argue such contribution is important as nested Monte Carlo simulations accommodates several large scale computations in financial services that are crucial in practice.