Yes it could be used in the argument against the contractors but that was only heard between your parents and the person who dropped off the cheque. It would then be up to the adjudicator to decide which is more "believable", 500 bucks for mere noise because they're working behind the house, or to use some section of the lot for a period of time, and I'm buying the latter.
Op is doing the right thing by just letting it rest since the deposit of the cheque pretty much sealed the deal and to fight it would mean spending money on what's just a temporary nuisance anyways. Take it as a lesson learned in that there is no free lunch in this world.

Since it's a temp fence I wouldn't be overly worried about it, although the company does sound shady. I'd be more concerned with what they do in the end, be sure they rebuild the fence in the right spot and don't fuck anything up.