Semiconductor optoelectronics are omnipresent in our daily life: from highly efficient LED lighting to high bitrate data transmission in the internet, it is an enabling technology. Future quantum technologies or neuromorphic computing will continue this trend. At the very beginning, sophisticated numerical models shall predict the optimum chip design in order to save technology resources and project record specifications. In this presentation, I will outline the specific multi-physics, multi-scale and multi-materials challenges we face in optoelectronics. Examples such as nanowire photovoltaics, vertical-cavity laser design, or ultraviolet LED research show the potential of numerical modeling, but also the challenge to capture the relevant physics arising from the coupling of optical, electronic, mechanical and thermal mechanisms.