In the ATK-Classical calculator, the boundary conditions are periodic for BulkConfiguration and a vacuum-padded cell for MoleculeConfigurations.
If you want to switch off periodic boundary conditions for a bulk-configuration in certain directions, you can increase the length of the cell vector in this direction to add a vacuum padding, which is larger than the interaction range of the potential, so that one side of the configuration does not interact with the opposite side.
If you want to impose more specialized boundary conditions, which e.g. modify the forces on the atoms (e.g. walls), you can do this using a hook function.