The Rebels, who finished November without a loss, put away the Scarlet Knights with a strong second half, exiting Tad Smith Coliseum with an 80-67 win.
With the win, Ole Miss moved to 6-0 on the year. The Rebels next travel to Middle Tennessee State next Saturday at 4 p.m.
"In order to have a chance to be the team that we want to be in March, you have to protect your home floor," Ole Miss head coach Andy Kenned said. "You have an opportunity to play a Rutgers. It's the first time Ole Miss has ever beat a Big East team in this building.
"It just shows that we don't get very many opportunities like this. We have to take advantage of it. It's a good start for us. You have to protect home."
Kennedy spoke last week after his team defeated Lipscomb of how he was eager to find out how his team would fair against better competition. The Rebels average margin of victory in their first five games was 33.2 points.
Welcome Rutgers, a Big East foe, who had four wins to one loss coming in.
The Scarlet Knights (4-2) jumped ahead early, leading by double-digit points early in the first half. Ole Miss couldn't get much going, especially from the free throw line, en route to a 36-32 halftime deficit. The Rebels made just 6 of 21 free throws in the first half.
The second half, however, was a different story. The Rebels rode the contributions of Murphy Holloway, Reginald Buckner and Marshall Henderson to a comfortable win, with the trio combining for 49 points and 29 rebounds.
"They're very consistent in what they do," Rutgers head coach Mike Rice said of Ole Miss. "They're very consistent in their approach. Their role guys understand their roles about as well as anybody I've faced in a long time, and they stick to that very well.
"That's hard to play against, because they really don't come outside themselves. They do what they do. They're going to play a physical, tough style of basketball."
Ole Miss outscored Rutgers, 25-12, to close things out after the game was tied, 55-55, with 7:54 to play. The Rebels used a 14-2 run to grab a lead it would never relinquish.
"Came out second half ready," Holloway, who had his third double-double of the year with 17 points and 15 points said. "I think (the first half) was the worst half we've played since I've been here."
Henderson paced Ole Miss with 18 points on 4 of 13 shooting. He was 3 of 12 from 3, including back-to-back 3s to give Ole Miss a 61-55 lead. He added three rebounds and two assists.
Buckner finished with 14 points and 11 rebounds. He was key in the second half, scoring 10 points and grabbing seven rebounds.
"Coach played through Reg," Holloway said. "It worked out well."
But according to Kennedy, it was Ladarius White who was the difference. " I thought he was the MVP for us," Kennedy said. The sophomore guard scored 13 points off the bench on 5 of 6 shooting. He was 3 of 4 from 3.
"I just wanted to go out there and do whatever it takes for my team to win," White said. "If that calls for me to go out there and score baskets, I'll do that. I'll do whatever it takes to make my team win."
Ole Miss was 28 of 64 from the floor, good for 44 percent. The Rebels were 7 of 23 from 3 (30 percent) and 17 of 39 from the free throw line, for a 44 percent clip.
They squeaked out a win in the rebounding battle, 42-41, but dominated the offensive glass, with 16 boards compared to seven for Rutgers.
"The thing that stabilized us was that we were pretty good defensively," Kennedy said. "We continue to defend. We didn't give them a lot of easy looks. We were much more efficient (in the second half)."
Freshman guard Martavious Newby, who was averaging 6.0 rebounds and 3.6 rebounds per game prior to the game, didn't play against Rutgers because of a coach's decision … Henderson had three makes in 12 3-point attempts. In his last three games, he's made just 7 of 34 3s … Anthony Perez played eight minutes, but had no points and only one rebound.