I had the same thing happening at first. I have 20 characters and ended up making a spreadsheet that the characters were named across the top and down the side I put years in. Then I outlined birth, places they lived etc. A lot of work but that was the only way I could straighten out who was how old when and what happened to them that causes them problems or results in their personal flaws. I had them graduating from different parts of the Eastern US, going to college, then moving back home or settling in my story's village. Had to tie it all together somehow without making a mother of a five year old, ten when she gave birth. ha! ha!