We investigate microgrid management where the controller tries to optimally dispatch a diesel generator as backup to primary renewable sources while maintaining low probability of blackouts. Dispatch takes place at discrete epochs (15 min in our example), while balancing takes place continuously, so only probabilistic guarantees are possible. Moreover, the likelihood of a blackout during the next dispatch period is not available analytically and can only be estimated. We formulate the problem as stochastic control where the Bellman equation features local probabilistic constraints that lead to an implicit state-dependent admissible control set. To tackle this challenge we develop novel Monte Carlo based algorithms, in particular empirical simulation procedures for learning the admissible control set as a function of system state. We propose a variety of relevant statistical tools including logistic regression, Gaussian process regression, quantile regression and support vector machines, which we then incorporate into an overall Regression Monte Carlo (RMC) framework for approximate dynamic programming. Our results indicate that using logistic or Gaussian process regression to estimate the admissibility probability outperforms the other options. Our algorithms offer an efficient and reliable extension of RMC to probability-constrained control. We illustrate our findings with two case studies for the microgrid setup with time-stationary and daily-seasonal net load dynamics.