Party X elected with a manifesto saying definite Brexit.
X then announce Brexit delayed till the 2nd term for... reasons.
X lose next election to party Y.
Y stops Brexit, which had a mandate before even implemented?
Substitute some other policy instead of Brexit like privatising the NHS or something you'd vote against. Does that change your vote to YES?