Some 39 percent of respondents said pot should be legal for personal adult use. Forty-four percent of those surveyed said it should be legal only for medicinal use. Just 16 percent said it should not be legal at all.
The responses come as two states have legalized recreational marijuana, with more than 20 states and Washington D.C. allowing some medical use of the drug.
"It's just a matter of time before it's in more states," said Steve Pratley of Denver, a 51-year-old pipefitter who voted for legalization in Colorado in 2012.
Pratley, who did not participate in the Pew survey, agreed with 76 percent of respondents who said people who use small amounts of marijuana shouldn't go to jail.
"If marijuana isn't legalized, it fills up the jails, and that's just stupid," Pratley said.
Legalization opponents, however, drew a distinction between making pot legal for all and thinking that pot users belong in jail.
"It's an illegal drug, period. I don't see it spreading," said Laura Sanchez, a 55-year-old retiree in Denver who voted against legalization. She agreed that pot smokers don't belong in jail, but she disagreed with legalization.
"I've seen no proof that it's good for anybody," said Sanchez, who also did not participate in the survey.
The poll suggested that despite shifting attitudes on legalization, the public remains concerned about drug abuse, with 32 percent of those surveyed calling it a crisis and 55 percent of respondents viewing it as a serious national problem.
And a narrow majority, 54 percent, said that marijuana legalization would lead to more underage people trying it.
As for mandatory minimum sentences, public attitudes have been shifting for years.
In 2001, the survey was about evenly divided on whether it was a good thing or bad thing for states to move away from mandatory minimum sentences for non-violent drug offenders. In 2014, poll respondents favored the move by a nearly 2-to-1 margin, or 63 percent to 32 percent. The other 5 percent either didn't respond or said they didn't know.
Public officials are well aware of the public's shifting attitudes on drug penalties.
Just last month, U.S. Attorney General Eric Holder testified in support of proposed sentence reductions for some non-violent drug traffickers in an effort to reserve the "the harshest penalties for the most serious drug offenders."
"Certain types of cases result in too many Americans going to prison for too long, and at times for no truly good public safety reason," Holder said last month at the U.S. Sentencing Commission.
Drug legalization activists said the Pew results come as no surprise.
"We see a growing bipartisan recognition that mandatory minimums went too far and did more harm than good," said Ethan Nadelmann, head of the Washington-based Drug Policy Alliance, which opposes criminal penalties for non-violent drug users.
Marijuana legalization opponents saw signs of hope in the survey, too.
Kevin Sabet, co-founder of Smart Approaches to Marijuana, which opposes pot legalization, pointed to the fact that 63 percent said it would bother them if people used marijuana openly in their neighborhood.
"Saying that we don't want people to serve prison time for marijuana is very different from saying I want a pot shop in my neighborhood selling cookies and candies and putting coupons in the paper," Sabet said.
The poll of 1,821 adults was conducted Feb. 14-23. The survey had a margin of error of plus or minus 2.6 percentage points.
Kristen Wyatt can be reached at http://www.twitter.com/APkristenwyatt