France, who began their campaign with a 4-1 loss to Latvia on Saturday, survived a barrage of Finnish pressure in the first period, as the Leijonat - who were unconvincing in their opener against Austria on Friday - had 19 shots on target in the first period to France's one.
Parssinen, Harri Pesonen and Waltteri Merala were among the players to be denied multiple times by French goaltender Antoine Keller, who needed a touch of good fortune to maintain his magnificent display:
In true giant-killing style, France made Finland pay for their lack of a cutting edge - Parssinen got called for high-sticking five minutes into the second period, and while still a man up three minutes on, Yorick Treille's men took the lead.
Tim Bozon worked the puck across to Kevin Spinozzi in the centre, and though his effort was kept out by Emil Larmi, the rebound was tucked home by Tim's brother, Kevin Bozon.
France tried desperately to hold onto the lead until the second break, but Finnish pressure eventually paid with two-and-a-half minutes remaining.
With Antti Penanen's side camped in the attacking zone, Teuvo Teravainen took it upon himself to drift over to the left and take a speculative shot, which floated through the crowd of skaters and past Keller.
In the third period, Finland began to dominate the shot count once again, but France, who finished 14th at last year's World Championship, continued to soak up all the pressure.
Les Bleus, who continued to keep their discipline, found themselves with a man advantage midway through the period after Mikael Seppala was called for interference.
The powerplay was the invitation the underdogs needed to cause a huge upset - a minute later, Jordan Perret gathered a loose puck to keep an attack alive and centred to Tim Bozon.
The elder brother needed two bites of the cherry, as Larmi saved his first effort before he lifted the rebound over the stranded goaltender.
Late in the period, Finland were struggling to create more chances and were forced to pull Larmi. However, the decision backfired, giving Perret the chance to surely seal the win with an empty-netter, which he took in style in the final three minutes.
But the scoring wasn't done there - Finland continued without a goaltender and set up a grandstand finish in the last 90 seconds, when captain Mikko Lehtonen set up Tolvanen to halve the deficit.
Having held out for so long against the side ranked 11 places higher than them, France could not deal with the pressure at the death, and with 59:32 on the clock, Finland saved themselves with the same partnership - Lehtonen again found Tolvanen, and the Seattle Kraken man made it 3-3.
Finland had finally hit their stride and made light work of the extra period in the end against their deflated opponents.
It took only 84 seconds for the winning goal to arrive - this time Tolvanen turned the provider, laying the puck off to Parssinen, who jinked his way through the French defence and tucked the puck past Keller.
Finland are two from two, sitting in third place in Group A, a point behind Sweden, who they face tomorrow night. The small consolation for France is that they have their first point on the board and sit sixth.
Sunday's World Hockey Championship Roundup
Elsewhere today in Group A, Canada overtook Sweden at the top of the standings by thrashing Latvia 7-1 to earn the biggest win of the championship so far.
Eduards Tralmaks gave Latvia the lead seven minutes in, but that was as it as it got for them, as they had no answer to the Canadian onslaught, led by Travis Konecny, whose equaliser was one of two goals and an assist he picked up.
Sidney Crosby also finished on three points as he was on hand to assist Konecny, Macklin Celebrini and Kent Johnson in the rout.
Meanwhile, Slovakia bounced back well after their heavy opening day loss to Sweden by inflicting a second loss of the tournament on Slovenia, 3-1.
Two goals in quick succession midway through the first period were followed by Michal Kristof making it 3-0 just past the midpoint in the match. Ken Ograjensek pulled one back for Slovenia in the third period, but a late comeback failed to materialise.
Over in Herning, Denmark, the USA's perfect start to the 2025 World Championship continued with a 6-0 demolition of Hungary, in which Frank Nazar and Cutter Gauthier scored twice each and Jeremy Swayman registered a second shutout.
The Americans stay top of Group B on goal difference ahead of Germany, who also picked up their second win of the competition with a 4-1 win over Kazakhstan.
The Central Asian side went 1-0 up early on, but Germany soon took control, with Maximilian Kastner, Alexander Ehl and Lukas Kaelble registering two points apiece.
One point behind are Czech Republic, who edged out Norway 2-1 for their first win in regulation time. All three goals came in the third period, with David Pastrnak scoring the winner. Norway are one of three teams in the group yet to pick up a point.