The tributes from the nation's political leaders were deeply personal. Vice President Joe Biden said he remembered thinking of Inouye: "I wish I could be more like that man. He's a better man than I am."
Former President Bill Clinton described Inouye as "one of the most remarkable Americans I have ever known."
Inouye was the first Japanese-American elected to both houses of Congress and the second-longest serving senator in U.S. history. He was awarded a Medal of Honor, the nation's highest military honor, for bravery during World War II, including a heroic effort that cost him his right arm.
"They blew his arm off in World War II, but they never, never laid a finger on his heart or his mind," Clinton said.
Inouye became one of the nation's most influential politicians, playing key roles in congressional investigations of the Watergate and Iran-Contra scandals.
Obama told a story of taking a summer trip across America as an 11-year boy and spending the nights watching the Watergate hearings on TV with his mother. The president said that as the son of white mother and a black father, he found it captivating to watch Inouye.
"To see this man, this senator, this powerful, accomplished person who was not out of central casting ... and the way he commanded the respect of an entire nation, it hinted to me what might be possible in my own life," Obama said.
Hawaiian music played during the service, reflecting Inouye's love for his home state. Inouye was Hawaii's first congressman.
One speaker after another described Inouye as a political giant with refreshing humility.
Obama said that Inouye ultimately decided he had done OK in representing his state and his nation the best he could.
"Danny, you were more than OK," Obama said. "You were extraordinary."
As a legislator, his specialty was steering federal money back home to help build the kinds of roads, schools and housing that Americans on the mainland took for granted.
On Thursday, colleagues and aides lined the Capitol rotunda five deep to say farewell.
The rare ceremony demonstrated the respect and good will he generated over the years. Only 31 have lain in the Capitol rotunda; the last was former President Gerald R. Ford nearly six years ago.
Inouye's body will be returned to Hawaii on Saturday.
AP White House Correspondent Ben Feller contributed to this report.