polystyrene “beans” will be better long term (they can get wet/washed). They’re basically solid plastic pellets.

Alternatively, dried beans were the OG for any sort of bean bag. (But they can’t get wet.)

The other thing is to go down a size on the hook so your stitches are tighter. As small as you’re comfortable with.