Life expectancy around the year 1800 was only about 36 years, so "middle age" (or median of lifespans) would be only about 17 or 18 years of age. By the time young people reached their late teens, they typically were expected to be working, married and carrying adult responsibilities.
Those life expectancy numbers can be a little misleading, though. A main reason that average life expectancy was only in the mid-30s is because infant mortality rates were very high. For persons who survived their childhood into adulthood, there was a good chance they lived to a much older age than 30-something.
