Chaffetz said throughout the Obama presidency, no one has been held accountable for failures, and that should stop.
"When the President makes a commitment to America's veterans, his staff should take that commitment seriously," Chaffetz wrote.
"The systemic problems at the VA call for new solutions and new leadership," he wrote. "Federal agencies need to be put on notice that incompetence will neither be rewarded nor tolerated. It's time to replace Secretary Shinseki and move forward with real reforms and proven solutions."
On Wednesday, Utah Democratic congressional candidate Doug Owens said President Barack Obama should demand Shinseki resign, saying the department is in need of sweeping reform and Shinseki is not up to the task.
Dave Hansen, campaign manager for Owens' opponent, Mia Love, said the veterans issues need to be fixed, but shouldn't be politicized. He did not say Shinseki should be gone.
Utah's former state veterans director, Terry Schow, disagreed with the calls for Shinseki's dismissal, calling his retention for the remaining 2 ½ years of the Obama administration the best chance to fix the problems.
"I'm disappointed," he said of Chaffetz's position. "None of the rest of the delegation has opted for that.… There's a lot of frustration out there and I get that but calling for him to resign is not the answer."
Schow, who serves on a state commission for veterans, has stayed in contact with delegation members and spoke with Sen. Mike Lee, R-Utah, on Thursday.
"I thanked him for not jumping on the bandwagon to get the secretary fired," Schow said, adding that it would take up to a year for a replacement to get up to speed in running the department.
"It's almost a frenzy, it becomes almost hyper-partisan," Schow said of calls for Shinseki's ouster, quickly adding "I'm not saying that of Chaffetz."
Shinseki's biggest political problem at the moment might be Democrats.
The Washington Post reported that, as of Thursday, 11 Senate Democrats had asked Shinseki to resign, along with two House Democrats deemed vulnerable in November's election.
The top House Republican, meanwhile, Speaker John A. Boehner, R-Ohio, said Thursday morning that he would "continue to reserve judgment" on Shinseki, the Washington Post reported.
"Is him resigning going to get us to the bottom of the problem? Is it going to help us find what's really going on? And the answer I keep getting is no," he said.
Dan Harrie contributed to this report.