We are naturally talking about substituted 4-Hydroxy-tryptamines here. Something of great importance is that "acetoxy" and "phosphoryloxy" are the respective
esters of acetic acid and phosphoric acid attached to the phenolic -OH of the tryptamine. When we have a benzyloxy group, we haven't got an ester, but an
ether (the one stemming from benzyl alcohol).
That is what makes it so hard to cleave compared to the others. If we had the analogous ester instead (a 4-
Benzoyloxy-tryptamine), then it would much more likely cleave in the body.
Look at the structures here and compare them. The two on the left both has the Ph-CH
2-O- grouping in common, but the two on the right has the same structure of their "connection" to the 4-oxygen. In this case it is the connectivity that is the most important thing, and not what kind of structure is sticking out.