Johnson shot 7 of 11 from the field on a variety of impressive moves around the basket. Sword was 0 of 7 from the field, but was 15 of 18 on free throws.
Mississippi State (11-4, 1-1 SEC) led 43-36 at halftime, but Ole Miss rallied to push ahead 58-54 by midway through the second half.
Jarvis Summers led Ole Miss (10-5, 1-1) with 23 points, including a 3-pointer that tied the game at 72 with 1:19 remaining. But Mississippi State scored the final four points at the free-throw line to seal the win.
Mississippi State has won 15 of the last 16 against Ole Miss in Starkville.
Ole Miss was playing without suspended leading scorer Marshall Henderson for a second straight game.
Mississippi State was helped by a huge 42-12 advantage on free-throw attempts. The Bulldogs made 28 of 42 (66.7 percent).
The Bulldogs fell into an early 10-5 hole, but quickly rallied for a 43-36 halftime lead.
Colin Borchert and Johnson both scored 11 points in the first half. It was another good game for Borchert against the Rebels — the 6-foot-8 forward scored 21 in a win against Ole Miss last season. Borchert finished with 14.
Mississippi State and Ole Miss both shot 13 of 32 (40.6 percent) from the field in the first half, but the Bulldogs built their lead thanks to a 21-5 advantage in free-throw attempts and the Rebels' 11 turnovers.
It was the Rebels' second straight game without Henderson — who missed the game because of a suspension announced in October — and the offense was inconsistent without him.
Derrick Millinghaus made three early 3s and Summers added two from long range to keep Ole Miss close in the first half. Then the Rebels went on a 12-0 run over a 4-minute span midway through the second half that turned a 54-46 deficit into a 58-54 lead.
The Ole Miss bench was crucial during that span. Anthony Perez made five straight points and also blocked a shot while Dwight Coleby grabbed an offensive rebound and slammed it home.
Ladarius White scored 16 points and Millinghaus added 15 for the Rebels.