Taking the advantages of tracking discontinuous material surfaces explicitly within the continuous mechanics framework, the phase field method has been successfully applied to study crack propagation and damage in brittle materials. Considering that the complex micro-structural inclusion-matrix interaction dominates the damage nucleation for heterogeneous PBX, we develop a phase field inclusion model based on the Eshelby's inclusion theory, and carry out the numerical simulation study of damage initiation and evolution. The phase field energy consists of the elastic energy and the inclusion-matrix interaction energy. Combining with the Mori-Tanaka method, the effective elastic moduli of heterogeneous PBXs with different volume fractions are derived. For the proposed phase field inclusion model, the nonlinear debonding effects and damage distribution can be characterized by the phase field order parameter directly. It owns explicity in physical mechanism, and completeness in mathematical theory. We apply this model to compute the high volume fraction heterogeneous PBXs with typical circular and polygonal inclusions, and investigate the influences of loading, inclusion shape, volume fraction and computational parameters on the debonding mechanism. Numerical results indicate that the inclusion-matrix micro-structural evolution promotes interface debonding and the formation of macroscopic material failure, which coincides with experimental observations.