Teacher compensation schemes are often criticized for lacking a performance-based component. Proponents argue that teacher incentive pay can raise student achievement and stimulate systemwide innovation. We examine a group-based teacher incentive scheme implemented in New York City and investigate whether specific features of the program contributed to its ineffectiveness. Although overall the program had little effect on student achievement, we show that in schools where incentives to free-ride were weakest, the program led to small increases in math achievement. Our results underscore the importance of carefully considering the design of teacher incentive pay programs.