Leveraging the existence of a hidden low-rank structure hinted by the diffusive limit, in this work, we design and test an angular space reduced order model for the linear radiative transfer equation based on reduced basis methods (RBMs). Our algorithm is built upon a high-fidelity solver employing the discrete ordinates method in the angular space, an upwind discontinuous Galerkin method for the physical space, with an efficient synthetic accelerated source iteration for the resulting linear system. Strategies are particularly proposed to tackle the challenges associated with the scattering operator within the RBM framework.