This question is not an interview question, but a question usually asked by an
interviewer right when you see them. This is especially common if the building
location is hard to find. Here is another question you should tell exactly what
happened.
With the first answer, they will say something like, that's good. Then they
will move on to other questions. If you answer with the second question, they
will probably talk a little more about how he or she agrees that this building
is hard to find. Either answers are acceptable.