TL;DR: We propose SPaCE, a general approach that can be deployed for any agent with a value function and task instances identified by their context without any additional information. We believe this is an important factor in future applications, including AutoML where in-depth knowledge about target algorithm instances is not always as readily available as common in RL.