Recently, most works focus on synthesizing independent images, while for real-world applications, it is common and necessary to generate a series of coherent images for story-telling. In this work, we mainly focus on story visualization and continuation tasks and propose AR-LDM, a latent diffusion model auto-regressively conditioned on history captions and generated images. To my best knowledge, this is the first work successfully leveraging diffusion models for coherent visual story synthesizing.