We analyze the coherent dynamics of a fluxonium device (Manucharyan et al 2009 Science 326 113) formed by a superconducting ring of Josephson junctions in which strong quantum phase fluctuations are localized exclusively on a single weak element. In such a system, quantum phase tunnelling by occurring at the weak element couples the states of the ring with supercurrents circulating in opposite directions, while the rest of the ring provides an intrinsic electromagnetic environment of the qubit. Taking into account the capacitive coupling between nearest neighbors and the capacitance to the ground, we show that the homogeneous part of the ring can sustain electrodynamic modes which couple to the two levels of the flux qubit. In particular, when the number of Josephson junctions is increased, several low-energy modes can have frequencies lower than the qubit frequency. This gives rise to a quasiperiodic dynamics, which manifests itself as a decay of oscillations between the two counterpropagating current states at short times, followed by oscillation-like revivals at later times. We analyze how the system approaches such a dynamics as the ring's length is increased and discuss possible experimental implications of this non-adiabatic regime.