A visitor who wants to record will just have a second recording device. Asking nicely to leave the phone behind could only protect from spying the phone owner never intended.
I thought of that, but dismissed the idea due to reality.
In reality, deliberately taking a second recording device other than your phone, escalates intention and risk. If we assume the visitors were unaware of the phone rule before arrival, there's no chance anyone will bring a second recording device.
> A visitor who wants to record...
Will use the recording device they already carry.
If discovered, your phone can hide behind a million excuses. "whoops, I forgot to end my call" or "whoops, I forgot to shut down my streaming app." It can also hide its recording status when using certain covert or custom apps. It is the ultimate recording device able to operate in the open.
To be fair, most phones have invasive Google/Apple/Chinese OEM services that work with high privileges and are invasive on their own, without apps. The share of people using degoogled OSes is small enough to assume it isn't the case. And, to be fair, if someone has a degoogled phone, you'd probably know that by now (I say as also a degoogled user XD)
His mistrust might be about the phone's owner and the recording apps they might use, more than the smartphone itself.
Socially it's much safer to direct suspicion at "the smartphone" rather than the people invited to your house.