There is evidence in the Hobbit that he eats young orcs. While your orcs do not imply that they are babies, there is another hint in the Books:

‘Then why didn’t he track Bilbo further?’ asked Frodo. ‘Why didn’t he come to the Shire?’  ‘Ah,’ said Gandalf, 'now we come to it. I think Gollum tried to. He set out and came back westward, as far as the Great River. But then he turned aside. He was not daunted by the distance, I am sure. No, something else drew him away. So my friends think, those that hunted him for me. 'The Wood-elves tracked him first, an easy task for them, for his trail was still fresh then. Through Mirkwood and back again it led them, though they never caught him. The wood was full of the rumor of him, dreadful tales even among beasts and birds. The Woodmen said that there was some new terror abroad, a ghost that drank blood. It climbed trees to find nests; it crept into holes to find the young; it slipped through windows to find cradles.

I guess with a little bit of imagination, it’s not that unlikely that he had eaten the babies of the woodmen of Mirkwood.

On top of this, he had a discussion with Sam about the best way to cook rabbits, and Gollum said:

They are young, they are tender, they are nice. Eat them, eat them!

So he likes his rabbits young! Which seems to be a preference of him.

But he definitely wanted to eat Bilbo. :D

He steals and eats orc babies.