I think it can be done alone, if you get behind the Phantom (or manage to stay in front of it, like Pirate Rose said) so that it can't damage you and there are no ships to bother you. The particular level of the Phantom shouldn't matter. But, you need a bit luck not to get other ships involved and it will take a while to get that ship down so it also demands patience.
I took an EITC Ogre (think it was level 34 and I was level 11, admitted, this is not the same thing, but it's just to illustrate you can take out high level ships with weak ships) down alone with my sloop and took no damage. Otherwise it was a stupid choice: hammering it down with unupgraded broadsides took over half an hour and the reward was 100 rep (which I would have gotten from a light frigate) and one normal cargo crate...
Simon