Before that, in high school, Brown played football, baseball and basketball, and was recruited in each of those sports, including Harvard (basketball) and BYU (football).
He chose basketball because a few glances in the mirror made one thing clear to him: "I didn't have the body to play football. I was a tall, skinny kid who liked to sit outside and shoot threes."
After bumbling around with his plans, telling coaches he was going on an LDS Church mission and then deciding against it, the basketball offer at Westminster became his best option. Brown took it and was mostly happy.
"It was small ball," he says. "But we worked hard and had fun."
Somewhere between studying finance and taking an internship at Huntsman Gay Global Capital, it dawned on Brown that the NBA wasn't in his future. When he received a full-time offer at the firm, he quit school, quit basketball and launched his career. When the firm closed its Salt Lake City office, Brown moved to an office in West Palm Beach. But after seven months there, in July 2011, he decided to return home to finish his degree.
He couldn't have imagined at that time his decision would bounce him back to South Florida to play pro football.
When Brown returned to Westminster, he went with his younger brother, Braden, an offensive tackle at BYU, to check out pro trainers who might be able to help Braden prepare for an eventual ascent to the NFL.
One of the trainers, Chad Ikei, took an interest in the basketball player/accountant/finance major who happened to be tagging along.
Says Brown: "He looked at me and said, 'You've got the rest of your life to work. You could be a great tight end.' "
As the weeks went by, Ikei pushed dreams on Brown he didn't even know he had.
"Chad was persistent," Brown says. "He wanted to take me on as a project."
After meeting with Ikei in Honolulu, when Brown traveled there to watch his brother play with BYU against Hawaii, he finally relented, quitting school again and moving to Oahu for full-time workouts, alongside 11 other NFL prospects.
Brown survived three sessions a day, six days a week.
"It was a three-month boot camp," he says. "It was a grind. By the time I left, I felt strong and fast, and I was up to 240 pounds. With a strict diet, I cut my body fat in half."
He returned to the mainland in time to participate in BYU's pro day on March 29, fully aware of his status and standing: "I was a no-name guy coming out of nowhere. I knew I had to turn some heads."
Brown did exactly that.
In the vertical jump, he went 39 inches, the highest of anyone at the event.
In the broad jump, he went 10 feet, 3 inches, the longest at the event.
In the 40, he ran a 4.43, the fastest at the event.
He ran routes and caught passes, impressing the scouts, all of whom asked the same question: Who is this guy?
"I was on Cloud Nine," he says. "It all hit me like, 'Wow, is this really happening?' "
Some scouts were skeptical, almost disrespected by the fact that Brown seemed to be toying with their game, walking in off the street, having played no college football, and putting up the numbers he did.
"A lot of it was mental," he says. "I'm normally a humble kid, but, going in, I said, 'I'm going to kill it. I'm going to be the guy the scouts are talking about.' "
Brown was right on both counts.
His brothers, Braden and Trevor, who is a redshirt freshman tight end at BYU, watched what their brother was pulling off and cried.
"It was an overwhelming feeling for all of us," Les Brown says. "When I quit school, some of my friends thought I was foolish, chasing some kind of high school dream. They thought I had a better shot at hitting the lottery than making the NFL. But I was doing it for me. I'd always wondered, 'What if I had played football?' "
The Eagles subsequently called, the Giants called, the Raiders and Colts called. The Dolphins, Packers and Eagles invited Brown to private workouts. He made it to Miami and then Green Bay before the Dolphins persuaded him to sign a three-year minimum deal with them, which included a modest signing bonus.
"All I wanted was a chance," he says. "When I started on this journey, I never thought this would happen."
It has happened. Brown will fly out Sunday, reporting to Dolphins mini-camp on Monday.
"I know the hard work is just beginning," he says. "I'm excited and humbled. I've got a lot of work in front of me. My goal now is to make the Dolphins look like the smartest team around for giving me a shot."
GORDON MONSON hosts the "Gordon Monson Show" weekdays from 2-6 p.m. on 97.5 FM/1280 AM The Zone. Twitter: @GordonMonson.