Simple, fast, and flexible risk aggregation in Hadoop

Value at risk (VaR) is a widely used risk measure. VaR is not simply additive, which provides unique challenges to report VaR at any aggregate level, as traditional database aggregation functions don't work. Deenar Toraskar explains how the Hive complex data types and user-defined functions can be used very effectively to provide simple, fast, and flexible VaR aggregation.

Value at risk (VaR) is a widely used risk measure that risk managers use to measure and control the level of risk that their firm undertakes. VaR is not simply additive; the VaR of a portfolio containing assets A and B does not equal the sum of the VaR of asset A and the VaR of asset B. This provides unique challenges to report VaR at any aggregate level, such as portfolio, desk, business hierarchy, or firm, as traditional database aggregation functions don’t work. Accompanied by a code walk-through and a live demo, Deenar Toraskar explains how the Hive complex data types and user-defined functions can be used very effectively to provide simple, fast, and flexible VaR aggregations. These functions, combined with other standard Hive features, can provide not only aggregate reporting but also other features such as slice and dice, rollups, and what-if analysis. Using Spark SQL’s external data sources, VaR aggregation and reporting can also be done over a business hierarchy sourced from a relational database. Deenar also shows how to use these building blocks to succinctly and easily implement the new risk measures introduced by the Basel Committee for Banking Supervision (BCBS) as part of the fundamental review of the trading book (FRTB), including expected shortfall, P-values, and market liquidity premia.