A new technique for analyzing modern genetic data suggests that prehumans survived in a group of only 1,280 individuals