Oblivious RAM (ORAM) and private information retrieval (PIR) are classic cryptographic primitives used to hide the access pattern to data whose storage has been outsourced to an untrusted server. Unfortunately, both primitives require considerable overhead compared to plaintext access. For large-scale storage infrastructure with highly frequent access requests, the degradation in response time and the exorbitant increase in resource costs incurred by either ORAM or PIR prevent their usage. In an ideal scenario, a privacy-preserving storage protocols with small overhead would be implemented for these heavily trafficked storage systems to avoid negatively impacting either performance and/or costs. In this work, we study the problem of the best storage access privacy that is achievable with only small overhead over plaintext access. To answer this question, we consider differential privacy access which is a generalization of the oblivious access security notion that are considered by ORAM and PIR. Quite surprisingly, we present strong evidence that constant overhead storage schemes may only be achieved with privacy budgets of epsilon = Omega(log n). We present asymptotically optimal constructions for differentially private variants of both ORAM and PIR with privacy budgets epsilon = Theta(log n) with only O(1) overhead. In addition, we consider a more complex storage primitive called key-value storage in which data is indexed by keys from a large universe (as opposed to consecutive integers in ORAM and PIR). We present a differentially private key-value storage scheme with epsilon = Theta(log n) and O(loglog n) overhead. This construction uses a new oblivious, two-choice hashing scheme that may be of independent interest.