We study numerically and analytically isolated interacting quantum systems that are taken out of equilibrium instantaneously (quenched). The probability of finding the initial state in time, the so-called fidelity, decays fastest for systems described by full random matrices, where simultaneous many-body interactions are implied. In the realm of realistic systems with two-body interactions, the dynamics is slower and depends on the interplay between the initial state and the Hamiltonian characterizing the system. The fastest fidelity decay in this case is Gaussian and can persist until saturation. A simple general picture, in which the fidelity plays a central role, is also achieved for the short-time dynamics of few-body observables. It holds for initial states that are eigenstates of the observables. We also discuss the need to reassess analytical expressions that were previously proposed to describe the evolution of the Shannon entropy. Our analyses are mainly developed for initial states that can be prepared in experiments with cold atoms in optical lattices.