1) The basis set / pseudo-potential you are trying use, is the preview version from ATK 11.8. There is included the complete set bundled with entire new basis set system. So you can now write
basis_set = [BasisGGAPBEHGH.Hydrogen_1_Tier_3, BasisGGAPBEHGH.Oxygen_6_Tier_5]
This means that you will use the HGH GGA-PBE pseudopotential for Hydrogen with one electron and the associate tier 3 basis set, and HGH GGA-PBE pseudopotential for Oxygen with 6 electrons and the associate tier 5 basis set. The higher the tier the better the basis set is. This is the new right way.
2) The HGH pseudopotential has been converted to UPF (since HGH does not contain an electron density), and therefore they have been encrypted. If you want to check that it is really the original HGH pseudopotential you can download it compare it like this:
BasisGGAPBEHGH.Hydrogen_1_Tier_3(NormConservingPseudoPotential('./1h1.hgh')
3) ATK-DFT "only" has localized basis sets, so if by basis set representation you mean localized basis set, then yes. There is no native planewave code in ATK-DFT. The result compare rather well with abinit. Here I have attached a small test example I did of Germanium using the same HGH pseudopotential.