Japan’s deputy prime minister apologized Tuesday and retracted comments that seemingly blamed women for the country’s declining population, the Guardian reports.

Speaking at a seminar in his constituency in southwestern Japan over the weekend, Taro Aso had said elderly people are unjustly blamed for the demographic challenges posed to the country of 127 million, including a declining and aging population.

“There are lots of weird people who say the elderly are at fault, but that’s incorrect,” Aso said, according to the Guardian. “Rather, those who aren’t giving birth to children are the problem.”

Japan’s population is projected to decline by almost 25% between 2015 and 2050, slipping below 100 million. Meanwhile the average number of births per woman has fallen from 2.0 in 1960 to 1.44 in 2016, according to the World Bank. By comparison, in the U.S. and the United Kingdom the rate was 1.8.

“The aging population, combined with the diminishing number of children, is the grave issue in the mid and long term,” said the 78-year-old deputy PM, who doubles as the Finance Minister.

His comments sparked criticism, including from MPs who accused him of insensitivity toward couples who cannot conceive. In response, Aso said his words had been taken out of context. “I’d like to withdraw my comments and will be careful with my words in the days ahead,” he said.

According to government statistics, the number of births in Japan fell to 921,000 in 2018, the lowest since records began in 1899. Last year was the third consecutive year the number of births dropped under 1 million.