The surface code is unarguably the leading QEC code, featuring a high threshold error rate ~1%, low overhead implementations of the entire Clifford group, and flexible, arbitrarily long-range logical gates - all despite only requiring a 2-D lattice of qubits with NN interactions. These highly desirable features come at the cost of high classical processing complexity. We show how to perform the processing associated with an n by n lattice of qubits, each being manipulated fault-tolerantly, in O(n^2) average time per QEC round. We describe how to parallelize the algorithm to O(1), using constant computing resources per unit area and local communication. Both complexities are optimal.