- There is limited knowledge on how to design effective interfaces to enable nonexpert users to interact with robot learning algorithms. This paper focuses on an interface design challenge: How to provide the user with sufficient information about the learned behavior. A simulated robotic task where the robot has online learning capabilities was developed. This platform was used to study the impact of the variability of the environmental conditions, the information provided on the relation between the learned robot behavior and the conditions in the environment, and the presence of a preview of the learned robot behavior on the use of the system and on task performance. The results show significant effects of the type and the quantity of displayed information. Forty-two participants made the best use of brief and contextualized notifications about changes in the environment: their presence improved the overall performance and the usage of the automation and reduced the workload. In contrast, adding previews of the learned behavior surprisingly impaired performance and reduced the use of automation.