The air con will only drop power when it is running. From my expierence, when the A/C is off, the loss is so small you probably wouldn't notice it. The biggest disadvantege would be extra weight and the A/C radiator infront of the main radiator.
*shrug*
I'll be putting the A/C in with my 20V swap, but thats just me.