His work is challenging, exhilarating and powerful to watch and to perform. Michael Ball, who has done plenty of both, pays tribute to a musical hero

The first time I encountered Stephen Sondheim was like everyone else: through snatches of old songs people performed in drama school, through Send in the Clowns, which everyone knew. I wasn't aware at the time that he was the writing force behind West Side Story and Gypsy. It often gets forgotten, because people think of Sondheim purely in terms of making difficult, highbrow music – which he did. But as a lyricist, he also worked on some of the most popular musicals ever.

I think Stephen himself would agree, somewhat, that he's been a tortured soul throughout much of his life, and has found it hard to be on a search for happiness. A lot of brilliant artists are conflicted in the same way and it informs their work. It's not an irony that he works in musicals; it's ignorant to believe the form only works on a cheery, superficial level. Sondheim has never written typical musicals – the kind made famous in the US in the 1940s and 1950s – he writes about the human condition, with layer upon layer of depth. His is musical theatre – like plays with music – not musical comedy, and there's a big difference. It's also why his legacy is so important: Stephen Sondheim changed the face of the medium.

Passion, which will be revived at the Donmar later this year, is a case in point. It's complex and bleak and not at all the sort of thing you'd expect to be set to music. I performed the lead in the West End for its UK premiere and worked very closely with Sondheim. He very kindly wrote, or fleshed out rather, the song "No One Has Ever Loved Me" for me, because there wasn't a big, 11th-hour song for my character Giorgio. We went into a little room and he gave me an hour's masterclass, talking me and teaching me through it. I can't quite explain why, but it was the most extraordinary session. Of course, I stayed in touch.

As a person, Stephen Sondheim is a very funny, very dry and very shy man. I've never witnessed any diva-ish moments, he just always seems so thrilled people are doing his work. The curious fact is that he and Andrew Lloyd Webber were born on the same day, almost two decades apart. There's always supposed to have been an intense rivalry between the two; it's said that Sondheim would have loved Lloyd Webber's commercial success and, likewise, Lloyd Webber would give his left leg for a share of Sondheim's critical acclaim. Whether we'll still be staging Sondheim in 500 years, like Shakespeare, is another matter. He would be appalled I'd even made the comparison, he's a very modest man. But yes, he's an innovator and he's stayed relevant throughout his career.

His masterpiece remains Sweeney Todd. It will always be my favourite. I can also reveal I will finally be performing it next summer in Chichester, fingers crossed with Imelda Staunton as Mrs Lovett. It's the work that took the artform to a new level. If you listen to the orchestrations, the lyrics, the witty rhymes, the dark aspects exploring the sinister vagaries of people's minds – it's incredibly powerful and original. You can see why Tim Burton was so drawn to it.

As a performer, once you've understood the genre of musical theatre, you can tire very quickly of the two-dimensional stuff. With Sondheim, it's always a challenge. It's difficult and exhilarating and he's so good on the complexities of relationships and on things going wrong. When the music is done right, it can only heighten that emotion and make the work much more powerful. Which is why Sondheim is for grownups and why his work is often so profound.