Falling fertility rates - the number of live births per woman go hand in hand with better education and career opportunities for women, their access to contraception and abortion and lower child mortality rates which mean women on average have fewer children.
For lower-income countries, a falling birth-rate could spell better living standards as a smaller number of children each get a bigger piece of the pie, whether that's health or education.
However, in countries where fertility rates have already been falling for years, shrinking further could cause problems. These countries will have to work out how to care for a growing older population, with fewer younger people to work as carers and to pay into the safety-net system.
A lot of the worries about caring for an ageing population assume everyone will be ill in old age.
But as well as life expectancy, the world has been making gains when it comes to "healthy life expectancy".
In pretty much every country around the world, with the notable exception of Syria, new babies are expected to spend more years in good health than those born in the year 2000 - five extra healthy years on average.
In Rwanda, the average baby has gained 22 additional years of expected life in good health since the start of the millennium.
In higher-income countries like the UK, Germany and the US, healthy life expectancy has increased by between one and three years.
"The fears around an ageing population have to be put into perspective," says Prof Sarah Harper at the Oxford Institute of Population Ageing.
"The health of older adults is already much better than it was," even a few decades ago, she points out, meaning older people can be "active, healthy" and paying in for a greater proportion of their lives. The elderly may well have to work for a lot longer. Governments are already raising the retirement age for pensions.
And, as Dr Hannah Ritchie at the University of Oxford's Our World in Data team points out: "We don't even know what the world of work will look like in 50 years".
Fertility rates and life expectancy are two parts of the equation when it comes to whether a population is growing or shrinking. The third is migration.
Countries that end up with much smaller populations of young people might want or need to attract young people from elsewhere. The world could become even more culturally and ethnically mixed, says Dr Ritchie.
No matter how big the gains in healthy life expectancy, the "oldest old" will probably always need care towards the end of their lives.
Dr Tiziana Leone at the London School of Economics warns countries with ageing populations face a crisis in terms of their health and social care systems.
We need to start now, by training the right workforce - "we'll need fewer paediatricians and gynaecologists", she says. Carers will be "as important as doctors"
A shrinking population is "a good thing" for the environment, according to Prof Harper.
But Dr Ritchie points out that economic growth is a stronger driver of climate change than population growth. If the world becomes richer and consumes more despite the numbers of people shrinking, environmental gains aren't guaranteed. Equally, while wealth and pollution have been linked over the past century, in recent years it's the richer countries that have been able to reduce their CO2 emissions by investing in technology. And this pattern could continue.