Twin suicide bombings in the Russian city of Volgograd left 34 victims dead last week, 400 miles from where the Sochi Games will be held.
No group has claimed responsibility for the bombings, but they highlight how vulnerable the Games could be to militant attacks. Separatists seeking to carve out an Islamic state in the North Caucasus have mounted attacks across Russia for nearly two decades.
"Russia has a special problem given the threats that have been leveled," Romney said. "There's no question about it. It's very, very frightening to have any kind of Olympic event on your national soil."
Romney's criticism of Russia is hardly new. During his presidential campaign, Romney called Russia "our No. 1 geopolitical foe."
Picking up that criticism, Romney said Russian President Vladimir Putin has not done enough to pressure Iran to curb its nuclear program or push Syrian President Bashar Assad from power. He also criticized Russia for harboring Edward Snowden, a former National Security Agency contractor who leaked documents disclosing several secret spy programs, and for shifting away from democratic reforms that Olympic organizers sought to reward when they picked Sochi to host the Games.
"Were it my choice, I would not vote for Russia to hold the Winter Olympics or the Summer Olympics. But it's not my choice," Romney said.
Russia also has faced criticism over its new anti-gay law.
Asked during a "Fox News Sunday" interview whether Russia's behavior undercut the Olympic Games' spirit, Romney answered yes. He invoked the 1936 Berlin Games that took place under Adolf Hitler and his fellow Nazis.
"Hitler's presence there, that certainly undercut the Olympic message," Romney said. "And surely that can happen in our time, theoretically or specifically."