Gilman enjoys three libraries: the William Passano Library (Lower School), the Walter Lord Library (Middle School), and the Edward R. Fenimore Jr. Memorial Library (Upper School). The three have a combined collection of more than 40,000 print and non-print materials, including books, periodicals, Kindles™ and electronic resources.

Each collection is appropriate to the grades it serves, and all students receive instruction in the use of library resources. Library resources are open to all members of the Gilman community. In addition to research tools, a richly diverse literature collection and large periodical collection, the library department sponsors book clubs, Books and Bagels Club, book fairs, book swaps, author visits, and helps design the annual summer reading lists. The Gilman Archives collects and preserves a variety of resources related to Gilman history.