- This paper introduces a framework for robust speaker localization in reverberant environments based on a causal analysis of the temporal relationship between direct sound and corresponding reflections. It extends previously proposed localization approaches for spherical microphone arrays based on a direct-path dominance test. So far, these methods are applied in the time-frequency domain without considering the temporal context of direction-of-arrival measurements. In this work, a causal analysis of the temporal structure of subsequent directions-of-arrival estimates based on the Granger causality test is proposed. The cause-effect relationship between estimated directions is modeled via a causal graph, which is used to distinguish the direction of the direct sound from corresponding reflections. An experimental evaluation in simulated acoustic environments shows that the proposed approach yields an improvement in localization performance especially in highly reverberant conditions.