If the joystick on Adventure is scrolling / panning the map even through you are not touching it, go to Settings in the Controller app and adjust sensitivity – reduce it in the direction you are getting the unwanted movement until it stops.

Alternatively: Disconnect it from the battery. Let the capacitors discharge (30min). Delete/forget the pairing in the bluetooth settings. Uninstall the Controller app. Reinstall the controller app.