The old house in front of the new section was built in 1818 for Denny Hull, who donated land for the nearby church. Katherine Drier bought the house and made it a country retreat for avant-garde artists. She persuaded the town to reconstruct Long Ridge and West Redding roads, moving the intersection farther from her house. She joined the Long Ridge Woman's Club and was its president for a number of years. In 1940 the present complex at the back of the house was constructed as a museum for her collection. But the collection was given instead to Yale University. In 1943 the house was sold to the Jesse Lee Academy which operated until 1952. The property was then bought by William Wickes who used it as a research laboratory until 1975. It was then sold to The Institute for Children's Literature.