Nine days later but figured I chime in. Sensor dust on these is, as mentioned, a bugger. I'm assuming you have dust-removal enabled on startup, and/or you've executed the function manually, yes? (I'm pretty sure it's on by default but I can't really remember). I've gotten the one dust issue cleared (on my older GR) with a vacuum cleaner once. Many will be appalled and I'm sure it just redeposited it elsewhere in the body but whatever.
That said, since 2009 I've owned a GR1s and four digitals - GRDIII, GRDIV, GR, and GRIII and that one dust issue is all I've come across. I have always, since the GRDIII, kept mine in one of those cheap microfiber sunglasses pouches, the kind they give away with a drawstring at one end. Makes it easier to get in and out of a pocket (microfiber is smooth) as a bonus. Purely anecdotal, but after well over a decade and 4 cameras without a real issue I'm starting to wonder if this has been the difference.
In terms of batteries, the Li92B option is a good one. Iffy contact interrupting the power to cause a lockup isn't going harm the camera, but it could conceivably lead to an SD card read/write failure. I wouldn't use those (
EDIT: the 3rd party batteries. DB110 and Li92b are fine) you have if they seem to cause that issue with any regularity. DB110 types are cheap - or try another 3rd party type. (I've got them all over the place at this point since my only real gripe about the III is the heavy battery usage. I've had a lockup or two, but never like you're describing.)
The only true fault I've had in reliability was sand getting into the assembly of my GRDIII. That was a headache. It was also my fault, but it didn't take much seize the gears and cause a lens error. Wound up being an incredibly expensive and inconvenient lesson. Don't set your GR in sand to take Milky Way exposures on the beach!