Fine-Grained Dynamic Resource Allocation for Big-Data ApplicationsJ1
Many big-data applications are batch applications that exploit dedicated frameworks to perform massively parallel computations across clusters of machines. The time needed to process the entirety of the inputs represents the application’s response time, which can be subject to deadlines. Spark, probably the most famous incarnation of these frameworks today, allocates resources to applications statically at the beginning of the execution and deviations are not managed: to meet the applications’ deadlines, resources must be allocated carefully. Our paper proposes an extension to Spark, called dynaSpark, that is able to allocate and redistribute resources to applications dynamically to meet deadlines and cope with the execution of unanticipated applications. This work is based on two key enablers: containers, to isolate Spark’s parallel executors and allow for the dynamic and fast allocation of resources, and control-theory to govern resource allocation at runtime and obtain required precision and speed. Our evaluation shows that dynaSpark can (i) allocate resources efficiently to execute single applications with respect to set deadlines and (ii) reduce deadline violations (w.r.t. Spark) when executing multiple concurrent applications.
Sat 11 JulDisplayed time zone: (UTC) Coordinated Universal Time change
15:00 - 16:00 | A26-Bugs and RepairJournal First / Technical Papers at Goguryeo Chair(s): Davide Falessi California Polytechnic State University | ||
15:00 12mTalk | Simulee: Detecting CUDA Synchronization Bugs via Memory-Access ModelingTechnical Technical Papers Mingyuan Wu Southern University of Science and Technology, Yicheng Ouyang Southern University of Science and Technology, Husheng Zhou The University of Texas at Dallas, Lingming Zhang The University of Texas at Dallas, Cong Liu UT Dallas, Yuqun Zhang Southern University of Science and Technology | ||
15:12 8mTalk | Fine-Grained Dynamic Resource Allocation for Big-Data ApplicationsJ1 Journal First Luciano Baresi Politecnico di Milano, Alberto Leva Politecnico di Milano, Giovanni Quattrocchi Politecnico di Milano | ||
15:20 8mTalk | The Assessor's Dilemma: Improving Bug Repair via Empirical Game TheoryJ1 Journal First Carlos Gavidia-Calderon University College London, Federica Sarro University College London, UK, Mark Harman Facebook and University College London, Earl T. Barr University College London, UK Link to publication DOI Pre-print Media Attached | ||
15:28 8mTalk | FixMiner: Mining Relevant Fix Patterns for Automated Program RepairJ1 Journal First Anil Koyuncu University of Luxembourg, Luxembourg, Kui Liu Huawei Software Engineering Application Technology Lab, Tegawendé F. Bissyandé SnT, University of Luxembourg, Dongsun Kim Furiosa.ai, Jacques Klein University of Luxembourg, SnT, Martin Monperrus KTH Royal Institute of Technology, Yves Le Traon University of Luxembourg Pre-print | ||
15:36 8mTalk | IntRepair: Informed Repairing of Integer OverflowsJ1 Journal First Paul Muntean TU Munich, Martin Monperrus KTH Royal Institute of Technology, Hao Sun Unaffiliated, Jens Grossklags Technical University of Munich, Claudia Eckert Technical University of Munich | ||
15:44 12mTalk | DLFix: Context-based Code Transformation Learning for Automated Program RepairTechnical Technical Papers Yi Li New Jersey Institute of Technology, USA, Shaohua Wang New Jersey Institute of Technology, USA, Tien N. Nguyen University of Texas at Dallas |