What is clustered resource management and YARN?

satya - 7/14/2019, 11:30:36 AM

Yarn is a protocol and implementation on how to distribute work

It is a protocol to distribute work across a number of machines

fault tolerant


Worker nodes

Node managers

Resource managers

Applications will negotiate for running themselves on in a distributed fashion

satya - 7/14/2019, 11:30:57 AM

Yarn on Azure HDInsight for example

satya - 7/14/2019, 12:39:39 PM

Application Master

An app is submitted to Resource manager

Resource manager allocates an application master for that application on one of the node managers nodes

The application master will then negotiate the parallelism