What is the splitting metric used in decision trees that assesses the purity of nodes?

The Gini index is a commonly used splitting metric in decision trees to measure the purity of nodes. It quantifies how often a randomly chosen element would be incorrectly labeled if it were randomly labeled according to the distribution of labels in the subset. The value of the Gini index ranges from 0 (perfectly pure, where all elements belong to a single class) to 1 (perfectly impure, where the elements are distributed uniformly across the classes). When creating decision trees, the algorithm seeks to minimize the Gini index, leading to splits that result in stronger classification and more homogenous nodes.

While entropy is another valid measure of node purity, it is generally used in the context of the Information Gain metric. Variance is not applicable as it relates to continuous data and assessing the spread of values rather than pureness in categorical outcomes. Information gain, while important, is derived from entropy, not directly from the Gini index itself. Therefore, the Gini index is specifically tied to assessing node purity in decision tree algorithms.