Imām al-Albānī [may Allāh have mercy upon him] says in al-Ḍaʿīfah (2/332):

“The opinion which legislates pulling a man back from his line to stand with him is incorrect as it is legislation without textual evidence. This is impermissible, rather it is obligatory to join the row if it is possible, otherwise he should pray on his own and his prayer will be correct because {Allāh does not burden a soul with more than it can bear} [al-Baqarah: 286].”