A ceremony, attended by PM Shinzo Abe, was held at Hiroshima's memorial park before thousands of lanterns are released on the city's Motoyasu river, BBC reported.
The bombing - and a second one on Nagasaki three days later - is credited with bringing to an end World War Two.
But it claimed the lives of at least 140,000 people in the city.
A US B-29 bomber called the Enola Gay dropped the uranium bomb, exploding some 600m (1,800ft) above the city, at around 08:10 on 6 August 1945.
On that day alone, at least 70,000 people are believed to have been killed. Many more died of horrific injuries caused by radiation poisoning in the days, weeks and months that followed.
People across Japan have observed a minute's silence to mark the anniversary. In Hiroshima a bell tolled at 08:15 local time - when the US aircraft dropped the bomb that flattened the city centre.
Addressing 40,000 people who attended the commemoration ceremony at Hiroshima's peace park near the epicentre of the 1945 attack, Mr Abe called for worldwide nuclear disarmament.
He said that that atomic bomb not only killed thousands of people in Hiroshima but also caused unspeakable suffering to survivors.
"Today Hiroshima has been revived," the prime minister said, "and has become a city of culture and prosperity.
"Seventy years on I want to reemphasise the necessity of world peace."
Mr Abe and Hiroshima Mayor Kazumi Matusi were joined by US Ambassador to Japan Caroline Kennedy for the official ceremony of remembrance on Thursday, which included silent prayers, the release of doves and a declaration of peace.
Mr Matsui described nuclear weapons as an "absolute evil" while urging the world to put an end to them forever.
"To coexist we must abolish the... ultimate inhumanity that is nuclear weapons. Now is the time to start taking action," he said in his annual speech.
Later in the day, thousands of paper lanterns will be released on the city's Motoyasu River - symbolising the journey to the afterlife of those who died.
The anniversary comes as divisions in Japan rise over Mr Abe's bid to pass unpopular legislation to expand the country's military role worldwide.
