The attack happened shortly after a man was arrested in possession of a bomb, a witness told the BBC.
The blast took place in the village of Dakasoye, about 20km (13 miles) south of the provincial capital, Kano.
"We lost 21 people and several others have been injured," Muhammad Turi from the Islamic Movement of Nigeria (IMN) said.
Police said they did not know who was behind the attack, but IMN organisers blamed the Sunni militant group Boko Haram.
The procession - an annual event lasting seven days - continued after the blast.
"We are not surprised that we've been attacked because this is the situation all over the country. This will not deter us from our religious observance," said Mr Turi.
The procession travels from Kano to Zaria in neighbouring Kaduna state, where the IMN, the country's biggest Shia organisation, has its headquarters.
Security forces had been ordered away from the procession after clashes between pilgrims and the army left several people dead last year, including three sons of its leader Sheikh el-Zakzaky.
Most of Nigeria's Muslims are Sunnis and correspondents say there are underlying tensions between them and Shia Muslims.
Boko Haram's six-year insurgency has killed thousands and made more than 2m people homeless. The group condemns Shias as heretics who should be killed.
The militants have increasingly attacked civilian targets since being pushed out of territory they controlled by a military offensive.
Nigerian President Muhammadu Buhari has given the military commanders until next month to end the conflict, but there are fears bomb attacks may continue.