- We investigate the semantic foundations of a method for modeling agents as entities with a mental state which was suggested by McCarthy and by Newell. Our goals are to formalize this modeling approach and its semantics, to understand the theoretical and practical issues that it raises, and to address some of them. In particular, this requires specifying the model's parameters and how these parameters are to be assigned (i.e., their grounding ). We propose a basic model in which the agent is viewed as a qualitative decision maker with beliefs, preferences, and a decision strategy; and we show how these components would determine the agent's behavior. We ground this model in the agent's interaction with the world, namely, in its actions. This is done by viewing model construction as a constraint satisfaction problem in which we search for a model consistent with the agent's behavior and with our general background knowledge. In addition, we investigate the conditions under which a mental state model exists, characterizing a class of “goal-seeking” agents that can be modeled in this manner; and we suggest two criteria for choosing between consistent models, showing conditions under which they lead to a unique choice of model.