Ai is about fooling humans. To decide whether an Ai behavior is good, we compare it to an expected behavior. Let’s say we build an Ai that simulates a philosopher. If it interacts correctly, and gives great answers, people will deem it as good. More importantly, if it mimics human experts then industry voices which is experts-backed will deem it as good.
Let’s say we want to test whether or not Ais are sentient. What do we do? We’ll first define what it means to be sentient. We’ll find those behaviors which sentient beings express and compare it to the Ai. But, the problem is that developers will go through the same process i.e. finding according to what criteria people judge sentient beings and code the Ai to emulate those behaviors. Whatever ways of interaction you put as criteria, the Ai will try to emulate that. It’s exactly like an enhanced mirror.
But, we are not foolish enough to decide that a mirror is indeed a person because we can see a human over there with human-like interactions. Ai means Artificial Intelligence. Meaning it is a field to monkey human intelligence.