I think this is Out of Time (1990) by John Marsden, republished by Tor Teen in 2005.
The cover shows the tag "What if you could slide in - and out - of time?"
The time machine looks like a calculator:
He thought at first that he was looking at nothing more than a mathematical calculator. Mr. Woodforde had assembled all the pieces into a black case that resembled either a calculator or a remote control for a television. There were four panels of buttons, two aerials and a screen, but they were all on a small enough scale to be fitted into a hand-sized case.
Finding the battery:
He prised open the cover of the little battery compartment and pulled out the battery. Bright, bright silver, it was like no other power source he had ever seen. It was tiny; the size of a gambling die. And the shape of one. But it was surprisingly heavy, weighing more than all the rest of the machine.
It has a lot of power, and can't be recharged:
"I'll be," he muttered. "Talkative little fellow aren't you? Look," he said, "you don't recharge them. They're not like a battery. But on the other hand, you can't run them out either. Well, not that anyone's been able to so far. If you've been running your torch off it, then I guarantee you a few millennia of bright light yet. Matter of fact, you could keep this base lit up for way past your lifetime, just on one of these little beauties."