After applying the ballast dry and tidying off sleeper tops etc with a paintbrush, it's a good idea to lightly mist the whole lot with water/washing up liquid. I recommend using something like this:
...before dropping the dilute PVA mixture into the ballast with a pipette while it's still wet. That way the PVA runs into the ballast much more readily.

...before dropping the dilute PVA mixture into the ballast with a pipette while it's still wet. That way the PVA runs into the ballast much more readily.