In a moving hot zone model with uniform temperature and an infinite temperature gradient under directional annealing, effect of second-phase particles (SPPs) on growth of columnar grain microstructure in polycrystalline materials was studied with phase field method. It shows that SPPs inhibit formation of columnar grain structure and inhibitory effect increases with increasing volume fraction of SPPs and decreasing size of SPPs. Effect of SPPs on final grain radius follows Zener relation under condition of directional annealing. Volume fraction of SPPs and their dispersive distribution at grain boundaries in materials should be reduced as far as possible to obtain better columnar grain structures during directional annealing.