The battery in my Pokemon Emerald GBA game was dead.
I unscrewed the screw that holds the game's cover in place and then used a knife to pry the contacts from the dead battery. This took some effort as the contacts did not want to let go. I finally got the battery out and inserted the new battery. The new battery was quite a bit bigger than the old battery but it still fit. The cartridge does bulge a bit now, but it still fits in the console and still works. It also no longer puts up the warning that the internal battery is dead.
The only thing I would say to anyone attempting this style of repair is to make sure you have plenty of light.