Or remove turbo from below, save shifting diesel rail, injectors, valve cover and whatnot, but is still a faff.
From top:
Engine covers off, air filter hosing off, air filter out, turbo intake pipe out (turbo to maf, one bolt at maf end a swine to get to, I use a 5mm allen key stuck in between fan shroud and a adjustable spanner to turn it, but one the tension gone, can be turned by fingers), turbo compressor pipe off (turbo to cooler, aluminium head with turney flange retainer add 10mm bolt), vac hose off wastegate, oil feed pipe off (engine end only), turbo off manifold (3 bolts out via holes in floor of airfilter lower housing, need to remove 3 rubber bungs and the under clipped in heat shield piece first then use a 12mm multiplied socket)
Then from below:
Riase front or all of car (safely), front undertray off (if you have one, i dont

) aluminium tray / bash plate off, unbolt flex exhaust, undo oil drain pipe from hose, unbolt turbo bracket, slide turbo down and out.
Refit in reverse, and check exhaust manifold whilst your there, if original, it'll likley be cracked. Make sure you prime the new turbo well and leave oil drain pipe out until you confirm oil flow right through the new turbo.
It's a pig on your back, but gets easier with practice lol.
2005 (55 plate) x5 3.0d sport, auto trans, nav tv thingy, but no dsp.
Under a turbo swap presently (fingers crossed it fixes her!)