Yep, they are not really refered to as Muppets on Sesame Street, but they're all Muppets.
Actually Sesame Street is much older than the Muppet Show too. But he created the Muppets even before that, when he was doing another children's show, the name of which I don't recall atm, and I am too lazy to consult wiki right now.
He wanted to make puppets that could be played in such a way that the puppeteer remained unseen on TV, so that the puppets really came to life. He first did that childrens show, in which Kermit was 'born', after that sesame street, then he and his colleagues performed on shows like Saturday Night Live, to aim for an aduly audience, and finally they got the go ahead for the Muppet Show.
Oh btw, the little green fellow below is a Muppet aswell, as Jim Henson and Frank Oz created him. Frank Oz is the puppeteer and voice for all the films. Bet you;d never had guessed that.
Fred