The judges ordered that a ruling in favor of Salt Lake County be vacated and the original lawsuit dismissed.
The trio, identified as J. Bronson, G. Lee Cook and D. Cook, sued the Salt Lake County Clerk's Office in 2004 after being denied a second marriage license. A lower court rejected their claim that the denial violated their constitutional rights.
Their appeal targeted Utah's constitutional ban on polygamy, criminal sanctions for the practice and an 1878 U.S. Supreme Court decision prohibiting the practice.
But in its review, the appeals court said a Utah district judge should have dismissed the lawsuit at the start because the plaintiffs lacked grounds to challenge civil and criminal prohibitions of polygamy.
The appeal also failed to make an argument why Utah's civil statutes banning polygamous marriages should be invalidated, the court said.
The justices also said the appeal failed to explain why "Utah's refusal to give civil recognition to polygamous marriages should be held to contravene their constitutional rights."
The appeal did challenge the criminal prohibition of polygamy, the court said, but that argument faced "a litany of seemingly insurmountable precedential obstacles." The justices then referred to case law and congressional acts since the 1860s that have upheld government's interest in limiting social harm.
The Cooks and Bronson are independent fundamentalist Mormons and sought to enter a plural marriage in keeping with their faith.
Brian Barnard, a Salt Lake City attorney who represented them, argued that polygamy bans violate the free exercise of religion, privacy rights and rights of association.
The prohibitions force consenting adults who enter plural marriages to live in fear of prosecution and subject them to public censure, he said. Barnard did not immediately comment on the ruling.
The court said his argument of "fear of prosecution" failed, too. "The alleged credibility of plaintiffs' fear is contradicted by their repeated admission that 'Utah's criminal law against polygamy is not being enforced,' " the court said.
The ruling also noted the Utah Attorney General Office's stated policy of focusing on crimes within polygamous communities that involve child abuse, domestic violence and fraud, rather than enforcing the law against consenting adults.
And the county clerk, the court said, has no power to initiate a criminal prosecution.
Nancy Kemp, an assistant Utah Attorney General, hailed the ruling as a "good decision."
"We are glad to see they took seriously the argument regarding standing," Kemp said. The court also "made an excellent point when they talked about the fact that there is no immediate threat of harm here. No one threatened to prosecute these people criminally for polygamy."
The ruling makes clear that a credible threat of prosecution is one of the hurdles that must be cleared by any challenge of the state's civil and criminal bans on polygamy.
"I don't think we've seen that," Kemp said.