The soreness of your muscles isn't really a factor if they have fully recovered from the workout. Remember muscles grow when resting not while working out. Even when the soreness is gone your muscles are still recovering.
There are different views on how much you should rest your muscles, some say at least 2 full days of rest, others say a whole week, I think that both are right to some point. Everyone is different so you should see what works best for you.
I'd say give the muscle group that you worked at least 2 full days of rest.