Tiny humans with health issues

How do they grow up to become the banished humans hidden away in the hamlets the "normal ones" don't want talk about.

Are there at least nicer hamlets where they can go and be invisible to the rest of society?