David West scored 26 points, including 12 in the third quarter and four in overtime for Indiana. Danny Granger added 21 points.
The Pacers won their third straight game and will try to close out the Eastern Conference series Tuesday in Indianapolis and get out of the opening round for the first time since 2005.
"They did a great job of coming back," said Granger, who returned to the game in the fourth quarter after spraining his right knee in the third. "That was a momentum win. They had a lot of momentum going. We're lucky to get out of here."
Indiana coach Frank Vogel echoed that fortunate feeling, but said he also likes being in the spot his team is in now.
"Yeah, it's a good place to be," he said. "We feel like we can get a win on our home court. It's tough to get a road win anywhere against anyone in the playoffs. To come in here in a tough environment and get two, it just speaks volumes of our guys' resiliency."
Orlando had a final chance to tie the game in the closing seconds, but Glen Davis' fade away jumper bounced off the side of the rim.
Jason Richardson led the Magic with 25 points and Davis added 24 points and 11 rebounds.
The Magic now head to Indiana staring at the possibility of their second consecutive first-round postseason exit as they continue their tumble since Dwight Howard's season-ending back surgery late in the regular-season. Including the regular-season, Orlando is 5-11 without the all-star center.
Only eight teams have been able to wipe out 3-1 deficits in NBA history, the last being Phoenix against the Los Angeles Lakers in 2006.
"You wish just one of those shots could've dropped because I thought our guys worked really, really hard," Magic coach Stan Van Gundy said. "We had some really, really bad stretches in that game, but we kept coming ... We're down 3-1 and it's a matter of mindset and whether you think you're still in the series or not."
The Pacers started the extra period with six straight points, including four by West.
Richardson responded with 3 to make it 95-92 and Jameer Nelson fouled out Roy Hibbert with his three-point play following a Pacers' miss to tie it.
Hill hit two free-throws on the other end to put Indiana back on top, but Davis tied it again with a twisting layup.
After an Indiana timeout, Hill hit a floater from the wing, but Davis again matched it on the Magic's next possession.
The Pacers quickly pushed the ball up the floor and Nelson fouled Hill in the lane to set up his decisive free throws.
Hibbert finished with 14 points and 11 rebounds. Hill started slow, but was huge down the stretch and ended up with 12.
"I was just determined to try and win," Hill said. "My teammates believed in me, and I got to knock down a couple of big shots. I get to ride my teammates coattails and fill in a little bit."
All five Magic starters reached double figures, with Nelson adding 12 points and Ryan Anderson and Hedo Turkoglu chipping in 11.
"I got a great shot off. Just short," Davis said of his final attempt. "It was on line. It was right there, but it was short. Two inches or one-inch up and we'd probably be playing more overtime right now or celebrating ... It's a fight now. Fight to see another day. Another game. Can't worry about it. Can't let it affect us."
Van Gundy said before the game that he thought he left his starting unit on the floor too long to begin Game 3. He went to the bench for the first time with just over five minutes to go on Saturday.
But Orlando's second unit struggled to provide the same boost it had in the previous three games.
Still, with Anderson starting out with one of his better offensive games of the series, some rejuvenated play by Turkoglu and strong free throw shooting, the Magic were able to keep nipping at the Pacers in the second half and down the stretch
Things got a little testy early in the fourth quarter, when Orlando's J.J Redick picked up a technical foul after a post-play skirmish with Indiana forward Tyler Hansbrough.
There were also a handful of foul calls that the Magic players took issue with and seemed to play into their frustrations as the Pacers built an 82-63 lead.
Orlando wasn't done, though, and used a 14-0 run to cut it to 82-77 with 4:40 to play, prompting the second Pacers' timeout in two-minute stretch.
Richardson nailed a 25-footer to trim it back to five and it was 89-86 after two Davis free-throws.
That was still the score when Redick got free on an out of bounds play and hit a 3 from the wing to tie it with 38.7 left.
A desperation 3-pointer by Hill with the shot winding down resulted in a shot clock violation and gave Orlando the ball with 14.7 showing in the clock, but Nelson's fade away jumper in the lane fell short at the buzzer.
West said after sweating out Saturday's win, they are going back home focused, but mindful that the Magic aren't about the lay down.
"It is hard to win games period," he said. "We will take the win. I thought it was good from the perspective that we made a lot of mistakes ... We were still able to respond and come out of here with a win."