Kollmorgen threw for 265 yards and three touchdowns for the Panthers,who are regarded as one of the top FCS teams in the country.
New Badgers starting quarterback Danny O'Brien completed 19 of 23 passes for 219 yards and a pair of touchdowns to Jared Abbrederis.
O'Brien is a transfer who was eligible to play for the Badgers right away because he already had graduated from Maryland, the same rule Russell Wilson took advantage of to play for the Badgers last year.
Coaches and teammates went out of their way this week to establish realistic expectations for O'Brien and shield him against direct comparisons to Wilson a remarkable playmaker who led the Badgers to a Big Ten title and Rose Bowl appearance last season, and now has won a starting job with the NFL's Seattle Seahawks as a rookie.
For Ball, it was a chance to brush off his recent off-the-field trouble. Ball was beaten up in an attack near campus a month ago. He sustained a concussion in the attack and missed part of camp.
It also was the debut for Badgers offensive coordinator Matt Canada, who replaced new Pittsburgh coach Paul Chryst.
Canada made his intentions clear right away by attempting to wear down Northern Iowa with a heavy dose of Ball. But the Panthers were up to the task early on. They held Ball to two yards or fewer on six of his first 12 carries.
The Badgers handed the ball to Ball eight times in a 14-play opening drive that ended with a 32-yard field goal by Kyle French.
O'Brien nearly got off to a rough start when standout Panthers cornerback Varmah Sonie jumped a route and came close to picking O'Brien's second pass attempt.
The Panthers' defense then made a stand deep in its own territory by stuffing Ball for a 1-yard loss on second-and-2 at the 14. The Badgers had to settle for the field goal after an incomplete pass that tight end Brian Wozniak appeared to drop.
Wisconsin continued to put together long strings of possession but couldn't turn its possessions into points. It had to settle for another field goal midway through the second quarter, a 35-yarder by French for a 6-0 lead.
The Badgers finally broke through late in the second quarter. Facing third-and-22 after a sack and a negative run by Ball, O'Brien hit tight end Jacob Pedersen for a first down. Ball ran for 14 yards through a big hole up the middle three plays later, and O'Brien found Abbrederis wide open in the end zone for a 10-yard touchdown and a 13-0 lead with just less than a minute left before halftime.
Wisconsin seemingly took control of the game on its first drive of the second half. O'Brien drove the Badgers for a 1-yard touchdown run by Ball. French's extra point was blocked, and the Badgers led 19-0 with 10:22 left in the third quarter.
The Panthers' offense finally got past the 50-yard line late in the third quarter and kept going from there. Kollmorgen threw a 21-yard pass to Brett LeMaster on fourth-and-4, then threw a 2-yard touchdown to Chad Owens on third-and-goal to cut the Badgers' lead to 19-7.
But O'Brien answered almost right away by throwing a rainbow to Abbrederis for a 53-yard touchdown and a 26-7 lead with 12:31 remaining.
The Panthers weren't finished. Kollmorgen threw to Johnson for a 55-yard catch-and-run for a touchdown to cut the lead to 26-14. Then they scored again when Kollmorgen found Johnson wide open down the right sideline for a 31-yard touchdown to cutting the lead to 26-21 with 7:24 left.
Northern Iowa forced a punt, and Wisconsin needed the big stop from Hemer to preserve the win.