Continual learning for robots refers to the continuous acquisition and integration of new knowledge and skills into their existing abilities. Similar to humans who learn throughout their lives, continually learning robots can adapt to new challenges and remain effective in evolving environments. In this talk, I will share work from my PhD thesis, which proposes novel continual learning methods for robotic manipulators, with a focus on real-world applicability. The discussion is divided into two main parts. The first part examines continual learning of dissimilar tasks, encompassing fundamentally different manipulation skills with varying objectives. Specifically, I will introduce our approaches to 'Continual Learning from Demonstration' where a robot learns a sequence of manipulation skills from human demonstrations. I'll delve into the methodology, exploring aspects of efficiency and motion stability and highlighting the pivotal role of stability in enhancing continual learning performance. In the second part of the talk, I will explore continual learning in dissimilar environments. This scenario includes situations where the task objective remains consistent, but the operating environment for the robot undergoes changes, necessitating adaptation. Here, I will present our research on 'Continual Domain Randomization', a method wherein a robot is trained using continual reinforcement learning across a series of diverse simulation environments, enabling zero-shot transfer and stable performance in the real world.