The congressman is the chairman of the House national security subcommittee and he's planning a border security hearing for May, where he'll continue to explore the topic.
"I really worry that there is no consequence of coming across the border illegally," he said. "There is no pain involved in that."
U.S. Customs and Border Patrol last beefed up punishments in early 2012 with a goal of reducing the number of immigrants who are returned to the same area of Mexico from which they crossed over. Instead, some were bused or flown hundreds of miles away and released.
Gregory Chen with the American Immigration Lawyers Association says there is more than sufficient punishment for those caught crossing the border. For one, it is a crime, where the punishments are ratcheted up for repeat offenders and border agents have wide discretion to immediately deport people without a legal review. He said the current system runs the risk "of eroding fairness and due process in the system."
And Chen asked: "What are we really trying to achieve here in terms of adding more deterrent to the border crossing issue?"
What Chaffetz wants to stop is not just an influx of migrant workers looking for a payday, but what he now sees as a sieve attracting immigrations from all over the world.
He went to the Eloy Detention Facility where more than half of the people held were from countries other than Mexico. Nine Romanians were captured by agents while Chaffetz was there.
He said agents told him that immigrants often claim political asylum and are allowed to stay in the country legally until they get a court date and that can take a long time.
"They are just gaming our system, knowing that they can get here and milk the system for years with no consequences," he said.
Chaffetz said incarceration is one option, but it is costly in terms of money and manpower. He'll explore other options going forward and wants to learn more about how people from countries all over the world are trying to sneak across the border with Mexico.
Border security will be a pillar of a larger immigration reform effort that Congress is expected to undertake this month. Other aspects will include a way for businesses to verify the legal status of workers, a revamped visa program that simplifies the process for immigrants to enter the country, a guest-worker program and an eventual path to citizenship for those who are now in the country illegally.
Chaffetz is willing to work with Democrats on the first four items, but opposes offering legal status or citizenship to immigrants who are here illegally.