Brian, that oil burning at the bottom of the turbo exhaust housing was also a problem I had at the time my oil drain was inadequate and it was blowing oil smoke.
You are correct in that it is the oil getting past the seals and then it is simply leaking out the bottom where the exhaust housing meets the turbo core.
This problem was completely fixed for me when I re-setup my oil drain as I mentioned before.
I am running oil drain line with an ID of 19mm (was previously about 10mm ID). I have the correct oil restrictor in the feed line and my crank case ventilation runs to a catch can and then the gases enter the system again immediately after the airfilter.
Make sure your crank case ventilation is NOT attached to the manifold or anywhere that gets positive pressure as this will mean that when ever the car is on boost (assuming you have a valve in the venting line) the crankcase will build pressure, which will also create pressure in the turbo oil system. Even worse would be the same scenario but with just a hose from the cam cover to the manifold which means your engines crankcase will see boost!! Very bad and I have seen peoples cars set up like this.
All I know is that I had the same problem as both you guys, on the same engine and after some minor alterations to the setup it fixed the problem 100%, my turbo was rebuilt about 3 years ago too so it's probably not as fresh as yours.
I might try and get some pics to show you my setup. There are a few 4AGTE's in Canberra that are setup basically the same as mine and no one has any probs with oil smoking turbo's so it must be ok.