I'm also worried that if we go with his plan we will officially have had a dispute and have to declare it if we ever came to sell, thereby potentially wiping thousands of pounds off the value of both of our properties.
I know a little, but as we always employ surveyors to do the dirty work so to speak I couldn't claim to have ever read the whole act (or to have understood it).
I think it's more the terminology of the Act rather than a blazing row kind of dispute, so while they'll be a record of an award if a future purchaser wants to check it, I can't think why it would affect the resale value, possibly if the owner wants to carry out future work? It's perfectly normal to withhold assent for works and use a surveyor to agree the process and in my opinion it's an eminently sensible thing to do if an your neighbour is ever proposing works to a party wall.
What you should be aware of is that you'll be liable for all the surveyors costs going forward.
Although it's often the case that each party appoints their own surveyor, an award can be drawn up by a single surveyor as they have to act independently in drawing up the PWA. I'd speak to your neighbour and ask if they'd consider going down this route as it will keep your costs down. Your architect should be advising you about the process in any case.