I think you can get "gimbal" style mounts that will do what you want as long as you adjust it right.
The potential difficulty here is that the central point you need to keep centered is *NOT* in the camera body, it's in the lens. If you have a lens that has a mounting collar on it (like a 70-200 f2.8 will) that collor will be positioned in the right spot on the lens that you want to keep steady.
It's the point in the lens where the light coming in crosses from one side of the centre line of the lens to the other... there's a technical term for it, but I cannot recall it right now.
(Focal Point??)
With a good gimbal or panoramic mount, you can mount the camera far enough "back" into the mount so put THAT point of your lens in dead centre up, down, left, right front and back. Then the centre will "hold" better.
Unless you want the central part of the flower to remain in essentially the same place in every shot, then the central point of the flower must be the point that everything revolves around.... and that would involve some significant euclidean geometry... or possibly a metre rule (or yardstick) that you can use to make sure you keep a straight line down the lens from the body to the central point of the flower. There will still be some "movement" but it should be lessened.
You could use a 4-way macro focusing rail, and something else on top of that like a gimbal, and lots of adjustments to make sure you keep the lines straight, and that you have reproducible positioning. (start at the centre point left to to right and the lowest point up & down, and use the rules on the macro focusing rails to determine how far left, right, up & down you move the camera for each successive shot, always making sure your central focus point in the camera is exactly on the centre of the flower, or using some very stable, solid stick/pole to check that you have things lined up correctly... doesn't sound like fun.
Some of the fancier gimbals and/or camera stabilisers might even be able to automatically keep the camera pointed at the same point for you... probably not cheap for something like that.
I hope that's intelligible!