Neem oil or an insecticidal soap is a good place to start. BUT I myself would get rid of the existing soil in the pot, spray off the roots well, so make sure there are no ant eggs or small ants left behind & then if all the roots look healthy I would repot in in the pot, after washing out the pot & rinsing it very well.
also you need to check the rest of the porch, to see where they came from. There may well be more.
I also think I would put some ant killer (such as Terro) near the pot, for just in case.
Of course you must keep it away from children or animals in some way, as it is poison.
But that is me. Hopefully someone will come along with a more concrete idea to help you. Do keep us informed.