This paper was presented at the 2006 Annual Meeting of the American Educational Research Association. The authors discuss some of the challenges and opportunities facing researchers when conducting randomized controlled field trials in school settings. The paper begins with a description of the current context for educational research, noting how researchers must strike a balance between rigor and relevance. It then describes one randomized field trial, set in New York City’s Administration for Children’s Services (ACS) childcare centers. Based on the project’s experiences in implementing and testing an early childhood mathematics intervention, the paper addresses the need to maintain the integrity of the rigorous research design, while dealing with a plethora of challenges that can pose a threat to the design’s integrity. It also describes the opportunities provided to researchers and practitioners. The paper concludes with a discussion of lessons learned that might generalize to other research project attempting similar studies. Implications for research and practice are described.