It comes from chapter 64 of the Tao Te Ching, the text traditionally credited to Lao Tzu. James Legge's 1891 translation reads: “the journey of a thousand li commenced with a single step.” A li is a Chinese measure of distance, so “a thousand miles” is a free rendering, and the line sits in a passage about acting while things are still small.
What it means
Every large undertaking starts with a small first action. In context the point is also about timing: deal with things while they are small and easy, before they grow.
Where it comes from
In Legge's translation the chapter opens: “That which is at rest is easily kept hold of; before a thing has given indications of its presence, it is easy to take measures against it … Action should be taken before a thing has made its appearance; order should be secured before disorder has begun.” Then come three images of great things with small beginnings: “The tree which fills the arms grew from the tiniest sprout; the tower of nine storeys rose from a (small) heap of earth; the journey of a thousand li commenced with a single step.”
So the famous line is one of a set of examples, and the lesson is less “just start” than “attend to small things, because everything large was once small”. Who wrote the Tao Te Ching, and when, is still debated; tradition credits Lao Tzu. Popular English versions vary in wording, but this line is genuinely in the text.