You could take all of the work and time you spent building a successful business. And just hand it to him so he can be a direct competitor, and likely undercut you until it’s not profitable to stay in business.

Or you could be polite, give vague noncommittal responses, and keep him as a client instead of a competitor.