The inmates had been moved to the Draper prison by that time, and most of the Sugar House lockup had been torn down. But some cells and the four stone guard towers remained.
Those 30-foot towers gave the place a "cold" feeling and made it an irresistible escape for kids in his neighborhood, he said.
"They were massive to a 5-year-old," Cheney said. "They were like something out of a movie."
Cheney is not the only Utahn who remembers the former state penitentiary, which closed in 1951. Florence Youngberg and Jose Corona have memories of setting foot in the prison when it housed more than 400 inmates.
Youngberg, 87, seemed less impressed with the place than Cheney, perhaps because she got used to looking out her window and seeing the 180-acre stone prison every day.
"I remember when we'd hear the bell," said Youngberg, who lived across from the Sugar House Prison more than 50 years ago. "That's when Mom would tell us to stay close because the bell meant someone had escaped."
Youngberg said she felt safe despite the holes the inmates would dig into the prison's almost-black fence. The only time she was frightened: When she visited the prison with her school choir to sing for the inmates.
The steel doors closed behind the high-schoolers as they made their way down silent halls until they reached their audience: a group of men dressed in white and black.
"It was a scary feeling," she said.
Youngberg left with her class shortly afterward, but the prison stayed with her, interesting her enough to write two books about it.
The lockup's "eerie" feeling also intrigued 78-year-old Jose Corona. He drove his father to the prison when he was 16 to visit an old friend.
But he forgot about the place until he returned from an Army stint and saw Sugar House Park. He didn't realize the park was the site of the former penitentiary until a friend mentioned it to him.
"I said, 'Shucks,' only it wasn't 'Shucks'; it was the other word," Corona said. "I just couldn't believe that the prison was here."
The state relocated the inmates to the Draper prison and shut down the Sugar House facility in 1951 to get them away from neighboring houses, Youngberg said. And, now, the current prison is surrounded by mushrooming subdivisions as well.
As for Cheney, Corona and Youngberg, they still frequent the Sugar House site, but now they do so to enjoy the sun, the grass and their families.
"It was a rough place," Corona said, "but they turned it into a nice park."