You could always borrow someone's dog or baby for a little while.
Or carry something else that's odd or interesting around. Speaking from experience, a yard-long plush beetle is a great conversation starter. (You do need a plausible reason for carrying it around though.)
Look for conversation starters on other people. An arm in a cast is obvious, but a nice pair of shoes, a book you've read or want to read or a good haircut work just as well. By asking someone where they bought their shoes or who their hairdresser is they will feel flattered and like you more, it's a classic 'in' to a conversation.
And of course everyone likes to complain. When the bus is late or the subway is crowded or the weather is miserable you have a given conversation topic.
What kinds of conversations are you looking for? To pass the time or to find new friends?