Arron Asham and Claude Giroux scored for the Flyers, who were outworked all night by the Maple Leafs.
Mayers and Asham exchanged goals in an evenly played opening period.
The Flyers, fourth in the Eastern Conference, are tied in points with the Pittsburgh Penguins with just over one week left in the season.
The Leafs started dominating play in the second period. Flyers goalie Martin Biron kicked out his left leg to stop Jeff Finger's long shot and was sitting in his crease when Stempniak whacked in the rebound at 1:54 as Toronto took a 2-1 lead.
With eight minutes left in the period, the Flyers iced the puck to relieve the pressure. Philadelphia coach John Stevens used his 30-second timeout, but the break didn't help.
White made it 3-1 at 13:03 with a blue-line blast that went over the kneeling Biron's left shoulder. Stevens pulled Biron and sent in Antero Niittymaki after Toronto took a 28-18 edge in shots.
Curtis Joseph, who will turn 42 this month, earned the win in his 10th start of the season.
With Stempniak penalized for holding a stick, Giroux smacked in a rebound on the power play at 17:09 of the third period to bring the Flyers within 3-2.
Philadelphia had a chance to tie it with a faceoff in Toronto's end with 25 seconds left, but Joseph went down to stop Jeff Carter's shot.