"There were times I wish I was home with them," he said. "You don't have that daily interaction, and that's kind of hard on families but you find ways to make up for it."
On the weekends, Stewart said his family hangs out, catches up and plops down on the couch to watch movies together. This summer, the Stewart clan plans to pile into the family's trailer to take rock-climbing trips.
And like most families, Stewart's brood is in constant contact thanks to modern technologies.
"Thank heaven for texting, because that's the best way to talk to teenagers," Stewart said, laughing.
Especially for a family scattered across the world.
His older three kids live on the East Coast Sean, 30, works in Washington for Sen. Mike Lee, R-Utah; Dane, 28, is in medical school at Virginia Commonwealth University; and Lance, 26, is in his first year of Teach for America in Jacksonville, Fla.
Back in Utah, son Brice, 18, just left on a Mormon mission to Italy, while his older sister, Kayla, 20, is flying off to London for her mission in the next few weeks.
Their youngest, Megan, 15, is a sophomore at Viewmont High School.
The congressman's wife of 30 years, Evie, acknowledged that having her husband spend half his time in Washington was a challenge, but a sacrifice she's happy to make.
"Whenever we sacrifice for our God or our country, whether it's in the military or public service or sacrificing for our family, it binds our hearts to them, and then we love our God and our family and our country more," she said.
While she says she hasn't gone to as many Washington social events as she would have liked, her first priority is being around for her youngest daughter, who, "deserves a mom," Evie Stewart said.
In addition to talking on the phone every night, the congressman says he hopes his wife will travel back to the Capitol with him once Megan graduates. Evie Stewart's next big trip to Washington will be later this month, when she'll join her daughters and her future daughter-in-law at the White House for the first lady's luncheon.
Despite the time away from home, Stewart says he knew what he was getting into.
"We knew it would be that way, and it's a small price to pay to have this opportunity," he said. "You can have your family separated and still keep a close family it just takes a little more effort."