Milan, riding his first Tour, looked to have been caught off guard when Australian Kaden Groves (Alpecin–Deceuninck) hit the front, but the Italian battled back to take the lead and held off the challenge of Van Aert, while Groves came in third.
Slovenia's Pogacar (UAE Team Emirates) remains 54 seconds ahead of Belgian Remco Evenepoel (Soudal–Quick-Step) in the overall standings.
Milan came close on stage three, beaten into second place by Tim Merlier, but this time the Italian, who has four Giro d'Italia stage wins to his name, powered home ahead of the experienced Van Aert, who was chasing his 10th individual Tour stage win.
"In the third one, we were pretty close to the victory," Milan said.
"We know that we just went a bit too early, but today we were really focused, we were really believing in it. My guys did an amazing job until the final. It was a really tough final."
After the last few action-packed stages, where the yellow jersey went back and forth between Pogacar and Mathieu Van der Poel, Saturday's race was expected to be a chance for the sprinters to return to the fore on the mostly flat 171.4km ride.
The peloton trundled along for the opening 90km, apart from a brief spurt at the intermediate sprint where Milan took maximum points, before French duo Matteo Vercher and Mathieu Burgaudeau (TotalEnergies) launched the first attack.
With several teams looking to set up their sprinters for the stage win, the break was never allowed to get too far ahead, though they managed to extend their lead to just over one minute with 20km remaining.
Burgaudeau went solo with 13km to go but was reeled in by the chasing pack after four kilometres and the tactical game of cat and mouse began.
Van Aert took the wheel of Groves as the Australian's Alpecin–Deceuninck team led the bunch into the final kilometre. Groves went early and did not have the legs to hold off Milan, while the Italian was able to withstand Van Aert's late burst.
Milan is the first Italian Tour de France stage winner since Vincenzo Nibali in 2019 and, having worn the green jersey on behalf of Pogacar in Saturday's stage, Milan is now the leader of the points classification.
Sunday's stage nine takes the riders 174.1km from Chinon to Chateauroux, a flat ride where the sprinters will again fancy their chances.