*** The problem is that PVA is slightly acidic and will react with some ballasts and minerals to change their colour. This is a fault in the ballast really as its intended application is as you are using it, and the Mfr should test before marketing.
Its not just ballast that can react to acidic glues... When I built my 4mm scale model of Turbomotive I was finalising detail the day before a modeling contest I intended to enter it in... I used real coal in the tender as always and tried a new glue I'd bought at a discount shop - imagine my horror when the whole coal load went a bright metallic emerald green.... really bright! This calmed down as it dried but remained looking like a dwarf emerald mine with sparkly green everywhere. A quick spray of black fixed it evebtually
Back to Ballast:
It is more common with real rock ballast than woodland scenics which is actually crushed nut shells, however the dye they use may be reacting with the glue.
Your fix is as follows:
either rip it up and do it all again... steering clear of cheap PVA and using a better quality glue OR copydex mixed 1/3 glue, 1/3 water and 1/3 meths.. even more meths/water thinner if its a heavy glue type.
Test a wee bit first.
Pre-wet the tidy ballast with a mist of Meths before dribbling on the Glue/Meths/water mix. Re-spray after dribbling on and it'll totally disapperar into the ballast.
Or:
Let it dry as is - totally - ie place a fan heater or just a fan so that it blows across the ballast and let it go for a very long time - then wait some more. Truly dry given what you have been doing is more than 48 hours, possibly a week total!
When its totally dry and cured, spray with a non water based paint, such as an auto undercoat or airbrushing gray undercoat, let dry thoroughly then wash over whatever stains you like, but do NOT use water based stuff - this will just reactivate the reaction.
Richard