With all the speculation here I figured it was worth looking into the origin of 'catch my drift' and its usage. Then it made simple sense...
Ok, you are the pilot on a sailing ship before electronic navigation. The pilot is steering a course in relatively calm waters and ties the steering wheel in place with a short length of rope and walks to the rail to smoke his pipe. He leaves a deckhand near the wheel to watch the direction of the ship relative to the night sky.
When the ship begins to drift off course, the deck hand 'catches his drift' and notifies the pilot - who immediately sets in a new course to compensate for the drift of the ship.
The key here is that 'drift' is not a mistake to the captian, it refers to 'real direction'. In heavy winds he might be pointed many degrees off course but the boat is actually moving (drifting) exactly where he wants it to go. So 'drift' means the actual direction of the ship's movement rather than the direction where the ship is pointing.

... Well, back to the present.. a speaker has a point he wants to make, the 'drift' of his speech/talk. The points he makes may use exmples from all kinds of seeming unrelated things but help establish the direction (drift) of his speech. A person who really understands what the speaker is trying to say really does "catch his drift" regardless of the wanderings of the exmples and points he makes along the way.

When someone says, "do you catch my drift?" they are not accusing you of being dense. They are actually saying "I know I probably used confusing examples and wierd ways of explaining myself, but underneath all of that was a main concept I was trying to get across. Do you understand what I was trying to say about that concept?" ...

So isn't 'do you catch my drift', a much easier way of saying all of that.. I think so. Now that I understand the main point of it, I am sure I will.. :)

In trying to research your question, its been sort of tricky. Not nearly as easy as I first thought. So far I've found this:

"Anybody know what CATCH MY DRIFT means and were it comes from?

It means, follow my train of thought, or understand the direction my thoughts are leading you. Usually expressed as a conditional, as in "if you catch my drift..." to see if you're keeping up with the tour group (if you catch my drift.) Experience has shown that this phrase, and its close relatives, are used most often by stupid and/or contemptuous people who find it hard to believe that mere You can follow the dazzling intricacies of their logic. If you catch my drift, if you know what I'm saying, if you see what I mean. "
from http://www.phrases.org.uk/.../2440.html

And someone tried to extend it further in this post on the same site

"It's very unlikely that anyone could find out who said it first. The phrase uses this meaning of "drift": "General meaning or purport; tenor" (American Heritage Dictionary, 1969, sense 5b of "drift" as a noun; by the way, sense 5a is "A trend or general bearing; direction," and from that we can see how "drift" came to mean the meaning that a speaker is aiming at).
The earliest quotation in the Oxford English Dictionary for this sense of "drift" is dated 1526: "Harde it is . . . to perceyue the processe and dryfte of this treatyse.""
located at http://www.phrases.org.uk/.../274.html

Doing a bit more digging reveals this:
"if you catch/get my drift (informal, informal)
something that you say to suggest that you have left out information or your opinion from what you have just told someone. 'She always has to be the centre of attention, if you catch my drift.'
See also: catch, drift"

And that is about all I could dig up after a good half hour of google searching. If you interested in other word/phrase origins, this page offers some good links and tips http://www.ipl.org/div/pf/entry/48468

------
Author Edit
------

I came across this later on looking up the origin of another phrase... From http://www.etymonline.com/index.php

drift
c.1300, lit. "a being driven" (of snow, etc.); not recorded in O.E., borrowed from O.N. or M.Du. drift, from P.Gmc. *driftiz, related to *dribanan "to drive." The verb is first attested c.1600. Sense of "what one is getting at" is from 1526. Drifter is first recorded 1864, as a mining term; meaning "a man following an aimless way of life" is from 1908. Driftwood first recorded 1633.

catch (v.)
c.1205, from Anglo-Fr. cachier "catch, capture" (animals), from V.L. *captiare "try to seize, chase," freq. of L. capere "to take, hold" (see capable). Sense shifted from original meaning of "chase, hunt." Past tense form caught is rare instance of Eng. strong verb with Fr. origin, probably infl. by latch, the cognate native verb, which this word replaced. Noun meaning "that which is caught or worth catching" (especially of spouses) is from 1596. Catchy was a colloquial word in 1831. To catch on "apprehend" is 1884, Amer.Eng. colloquial. To catch (someone's) eye is first attested 1813, in Jane Austen. Catchword (1730) was originally the first word of the following page inserted at the right-hand lower corner of each page of a book; extended to "word caught up and repeated" (especially in the political sense) by 1795. Catch as catch can first attested 1393.

In Drift it says "Sense of "what one is getting at" is from 1526." and in Catch it Says "Noun meaning "that which is caught or worth catching" (especially of spouses) is from 1596." So one could infer that someone as early as the 1500's could have coined this term.

Hope this helps.

Nahhh,...you're all over-thinking it,...it's a log-rollers phrase-my drift is the log I lost,...a "drift" is all my logs,...in the early days of west coast North American logging, people rode a "herd" of logs - a "drift" to market,...specialized log-rollers actually tended them down river,...a loose log was a drifter and a downstream man would be encouraged to "catch my drift",...BTW also the origin of the slang word "drifter",....