Wyler said the ashes from the weekend fire were still warm as he picked up the damaged pages of a medical book Monday.
The library was the brainchild of Stefanie Colgrove, who was raised in the area controlled by the FLDS, but does not follow the polygamous sect's faith.
"I was just thinking if there was something you could give to the community and help them realize life is going to go on and help them be more open to reality, I was thinking a library would do that," Colgrove told The Tribune in 2008. "It is like music, it spans generations and covers a lot of different backgrounds."
Elaine Tyler, of The Hope Organization, a nonprofit group that helped collect books for the library, estimated the damage at about $15,000.
"I think it's a hate crime," said Paul Murphy, a spokesman for the Utah Attorney General's Office. "The fact that so many people from the community and outside the community rallied together to get those books there to have someone come in and burn them is just outrageous."
Murphy personally delivered thousands of books to the twin towns of Hildale and Colorado City in 2008.
"I just remember an outpouring of people who really wanted to see something good for the community," Murphy said. "For someone to burn them, they should be ashamed. I'm hoping the police take it seriously and find out who did it and make sure they're held accountable."
A message left with the Colorado City Police Department was not immediately returned Monday evening.