The penalties for not reporting these accounts are stiff. If there is more than $10,000 in an unreported account, the IRS can impose penalties of 100,000 more if the accounts are really big.
The IRS announced a program Wednesday that would largely waive these penalties if taxpayers come forward and show that they didn't hide the money on purpose.
Americans living abroad can have all penalties waived, if they file three years' worth of tax returns and pay any back taxes.
Americans living in the U.S. can come clean by disclosing overseas accounts and paying a penalty equal to 5 percent of the account's assets.
For people who are willfully evading U.S. taxes, the IRS is tweaking an existing program that imposes stiff penalties for people who come forward, but allows them to escape criminal prosecution.
"Our aim is to get people to disclose their accounts, pay the tax they owe and get right with the government," IRS Commissioner John Koskinen said in a statement.
IRS officials declined to provide examples of how someone could inadvertently fail to report income or bank accounts.
Instead, they offered this definition: "Non-willful conduct is conduct that is due to negligence, inadvertence or mistake, or conduct that's the result of a good-faith misunderstanding of the requirements of the law."
"If a taxpayer is very conflicted about this, I think that what we would advise is that they consult with a tax adviser about the willfulness standard," said Michael Danilack, an IRS deputy commissioner.
Danilack said the agency expects thousands of people to take advantage of the program. In 2009, the IRS estimated that more than 7 million U.S. citizens live abroad, not counting military.
The disclosure program is a particularly good deal for Americans living in countries with higher tax rates than the U.S.
Typically, U.S. citizens living abroad must file a U.S. tax return and pay U.S. taxes on all income, regardless of where it is earned. However, they can deduct taxes paid to a foreign government from their U.S. tax bill.
The U.S. stepped up efforts to crack down on people who are willfully hiding assets overseas in 2009. Since then, the IRS has offered a series of programs to encourage people to come clean. In general, people who come forward can escape criminal prosecution in exchange for paying reduced penalties and back taxes.
More than 45,000 people have come forward so far, paying about $6.5 billion in taxes, interest and penalties, the IRS said Wednesday.
Starting next year, it will get even tougher for Americans to hide assets overseas. As part of a 2010 law, more than 77,000 foreign banks from about 70 different countries have agreed to start sharing detailed information about U.S. account holders with the IRS. The effort of part of a global crackdown on overseas tax evasion.
Follow Stephen Ohlemacher on Twitter: http://twitter.com/stephenatap