The scarf's absense is definitely noticed, and Mikasa has fully turned around where she's still standing on the step of the bus- essentially holding up the traffic and eliciting a series of complaints from those waiting behind her. From her vantage point, it's easy to see where the scarf blew off to, and she quickly excuses herself through the crowd, giving up her good spot in line to fall back where Toji stood.

"Sorry, that's mine." Unnecessary, since who else would push through a throng of hot, sweaty people to chase after a raggedy old scarf?