An obituary of Big Jim Sullivan, session musician, who managed exactly that. Here.
Not everyone can manage it, but good when it happens. Although there is something of a danger that an enjoyable hobby is reduced to the daily grind.
Should you always follow your passion? No. The fact that someone is passionate about golf, as an example, does not always mean that they are good at it. And of course whatever you are passionate about needs to provide you will enough income.