I guess maybe the "mysterious recluse" thing has some kind of princess-in-a-tower allure...?

What happens if you don't live up to the mysterious princess in a tower image?

If I were in that spot, I'd have politely declined his offer for a date. And just left it at that. You don't need to volunteer any info about not looking for a partner, your single-or-not status, or whether you'd date him if you were. If he is considerate, he can accept a polite "no, thanks".