Please note that the tick-boxes x,y,z don't have anything to do with the FORCE optimization; they are used to determine in which directions the UNIT CELL should be kept fixed, for the strain optimization. But perhaps that's what you did/know, I just wanted it to be clear.
In some sense it should matter too much exactly how you create the initial geometry, so yes I would say never mind the rotation, just move the atom out of the plane by some small (but not negligible) amount, and run the force optimization with a low force tolerance (and a low scf tolerance, since it can be hard to reach a very small force tolerance if the electronic structure is not really well converged).
In principle it's best to perform the combined force/stress relaxation, keeping z fixed for simplicity. Also, as you indicated, keep the atom at the origin fixed, it makes it easier to compute the bond length and angle later.
There is no built-in rotate method (hm, well, there is, but it's not official...), but it's also fairly easy to write yourself in Python. If you still feel this is important for you I can provide some code or the internal method to use (that depends on which version of ATK you use, however). However, since the exact bond length (on the level of accuracy desired) most likely will depend on the rotation angle, I'm not sure this method will be efficient; for each angle you would have to scan the bond length too. Granted, the calculation is quite fast, and it can be interesting to see where the force/energy minimum lies and how flat it is - my suspicion is that it's quite flat.
But start with the basic optimization, with low tolerance, high k-point sampling (9x9 at least!), and compare LDA and GGA. It will be interesting if you can share your detailed results with the community as you progress, like which is the optimal choice of k-points, etc.