Gatland: All to play for
Dan Biggar kicked four penalties and converted George North's crucial 47th minute try to pull Wales out to 19-3 before France skipper Guilhem Guirado crossed for a late consolation try to save his side's blushes.
The result left Wales well in the running for the Six Nations title, having opened with a 16-all draw with Ireland before seeing off Scotland (27-23).
Next up on March 12 is an away trip to England before a final home game against Italy.
"Victory keeps us in the championship. If we'd lost, we'd be out of it," Gatland said.
"There's still all to play for in two weeks.
"Our defence was outstanding, our set piece was good again and we were good in the air.
"It's going to be a really tough test match [against England] in two weeks' time.
"But Twickenham is a great stadium and we love playing there. We have to have the confidence and self-belief that we're able to do it."
Gatland said the victory over France, under new coach Guy Noves, came after a "really tough Test match".
"We talked at half-time about keeping our composure," the New Zealander said of the 6-3 lead the Welsh took into the second period.
"So going 19-3 up with 10 minutes to go we were very pleased with that."
But Gatland said a penalty count of 17, almost twice the normal count, was too high.
"A penalty count like that puts you under some pressure," he said. "It's frustrating, and it's something have to have a look at."
Guirado's late try, said Gatland, was "only the second try we've conceded against France in six matches".
"The most pleasing thing was our defence," he added.
Noves, who took over from Philippe Saint-Andre after France's disastrous World Cup campaign that ended in humiliation at the hands of New Zealand in the quarter-finals, admitted that North's try was "the big turning point of the match".
The Wales winger capitalised on a fantastic Jonathan Davies kick, showing great pace to get to the ball. But a comical moment saw North miss tapping the ball on, only for the covering Jules Plisson to toe-poke it into his path for a pick-up and slide over for a try.
"The biggest difference was on the counter-attack. For the try, it was because we gave away the ball," said Noves, who led Toulouse to four European Cup titles.
"That try allowed the Welsh to take confidence. It was a difficult way to concede a try, especially after the spirit we showed in the first-half to go into the break at only 6-3 down.
"After the try, the break was made. But the lads showed a desire to play, they stuck at it right to the end. They crossed the line two or three times."
Noves added: "We weren't surprised by Wales. We knew they relied a lot on the power of the backline and they have a strong kicking game.
"They put pressure on us and pinned us back in our half. They played simply but efficiently."
Next up for France is a trip to Edinburgh to play Scotland on March 13 before entertaining England at home a week later.
Agence France-Presse
