From here, you should be able to solder in the new replacement battery and reassemble the device

The new batteries should come already connected together as one unit and with the tabs needed to solder them to the board

If you are planning on making the battery yourself, I've included a few dimensions to help. I believe normal AAA batteries should fit, but I can't speak for their compatibility