When the little kittens are less than three weeks old, you are still stimulating them and teaching them to go to the bathroom. Later they will do it on their own, and believe it or not kittens will always find their litter boxes and learn to use them on their own.
It can help to put them into their litter box and move their little feet around but usually they will get the hang of it extremely quickly. If the mother is around she will do that for them but if not you can help your kitten out with that.
It is good to have a pan with a low side so that they can get in. Since pans come in many different sizes and heights. You can even just get a cooking pan and put litter in it. You can use what they call 'clay litter' this is perfect for absorbing a lot of fluids but you will have to sort out pick out the faeces with it. Otherwise you just mix it and let it dry up and absorb the liquid.
You could set up your litter pan in the bathroom because it is a nice small space and the cats can get familiar with it. Cats have a natural instinct for burying their faeces and their urine which works really well for humans because they also do not like to go, in the places they sleep or play.
Try not use the pine litter, which is bigger and chunkier for your kittens. Those are perfect for the grown up cats only and not effective for the smaller ones. Kittens like to really get engaged with their litter box.