We propose a method to study the thermodynamic behaviour of small systems beyond the weak coupling and Markovian approximation, which is different in spirit from conventional approaches. The idea is to redefine the system and environment such that the effective, redefined system is again coupled weakly to Markovian residual baths and thus, allows to derive a consistent thermodynamic framework for this new system–environment partition. To achieve this goal we make use of the reaction coordinate (RC) mapping, which is a general method in the sense that it can be applied to an arbitrary (quantum or classical and even time-dependent) system coupled linearly to an arbitrary number of harmonic oscillator reservoirs. The core of the method relies on an appropriate identification of a part of the environment (the RC), which is subsequently included as a part of the system. We demonstrate the power of this concept by showing that non-Markovian effects can significantly enhance the steady state efficiency of a three-level-maser heat engine, even in the regime of weak system–bath coupling. Furthermore, we show for a single electron transistor coupled to vibrations that our method allows one to justify master equations derived in a polaron transformed reference frame.