The ex-United man scored a last gasping header from Mesut Ozil’s free-kick to end his side the valuable three points.
The game was change after Jamie Vardy converted a penalty to beat Petr Cech when Monreal was penalised for bundling the Mr. Fox inside the box.
Arsenal though had too Penalties turned down after Leicester City players bundled the ball in two occasions inside their own box.
The game took a crucial turn in the 54th minute when Simpson was shown a second yellow card for tugging back Giroud, just five minutes after fouling Alexis Sanchez to pick up his initial booking – the first time Leicester had been reduced to 10 men in the league this season.
It was Arsenal’s first shot on target that gave them the equaliser. Hector Bellerin’s cross from the right was cleverly headed on by Giroud and Walcott beat Kasper Schmeichel with a cool finish nine minutes after coming on.
Arsenal pressed hard for a late winner, with Per Mertesacker heading inches wide and Giroud being thwarted by an excellent reflex save from Schmeichel.
And the breakthrough arrived when Ozil’s perfectly flighted free-kick was coverted by Welbeck, yet another substitute to send the home crowd wild.
This is not the first time Arsenal are winning game on stoppage time. Thanks to goals from the substitutes which move them within two points leaders Leicester City.