For clarification: I think you have the _unequivocal_ right to grab your rifle and chase both of the men off your property. I am saying that it would be uncharitable to do so to the second man. At least, until and unless you have found someplace else suitable for him to stay.
I think Christians can argue over what is "until" and what is "suitable", but we're clearly called to charity for our neighbor and this is a literal neighbor. We are obliged to treat him as we would want to be treated.
