Hahaha Anything overnight with a baby (especially if less than at least 5 days not including travel) always feels like a giant waste of time/energy/money/patience/etc. The amount of crap you have to take with you cannot be overstated.
I preferred to barely leave my back yard unless on foot.
That said, I'm not the only one in my family and so I've done trips like this a lot. The only way I've managed to make it reasonable is with a roof cargo box and a Tiguan.