We consider optimal control problems governed by PDEs with uncertain parameter fields, and in particular those with objective functions given by the mean and variance of the control objective. To make the problem tractable, we invoke a quadratic Taylor approximation of the objective with respect to the uncertain parameter field. This enables deriving explicit expressions for the mean and variance of the control objective in terms of its gradient and Hessian with respect to the uncertain parameter. The stochastic optimal control problem is then formulated as a PDE-constrained optimization problem with constraints given by the forward and adjoint PDEs defining these gradients and Hessians. The expressions for the mean and variance of the control objective under the quadratic approximation involve the trace of the (preconditioned) Hessian, and are thus prohibitive to evaluate for (discretized) infinite-dimensional parameter fields. To overcome this difficulty, we employ a randomized eigensolver to extract the dominant eigenvalues of the decaying spectrum. The resulting objective functional can now be readily differentiated using adjoint methods along with eigenvalue sensitivity analysis to obtain its gradient with respect to the controls. Along with the quadratic approximation and truncated spectral decomposition, this ensures that the cost of computing the objective and its gradient with respect to the control--measured in the number of PDE solves--is independent of the (discretized) parameter and control dimensions, leading to an efficient quasi-Newton method for solving the optimal control problem. Finally, the quadratic approximation can be employed as a control variate for accurate evaluation of the objective at greatly reduced cost relative to sampling the original objective. Several applications with high-dimensional uncertain parameter spaces will be presented.