Vision is also not totally a machine but has living human cells and a brain.
In the first Thor movie, Thor was not able to lift the hammer after he was banished to Earth because Odin felt that Thor was not worthy to use the hammer because Thor previously misused the hammer not to preserve lives but to misuse it to harm and kill people unnecessarily.
The worthiness to hold the hammer is to preserve life and does not come from other intentions.
Thor was able to get back his power and hold the hammer because he finally gains the intention to preserve the life of the people in the small town as seen in the following video.
To use the hammer with the intention to remove Evil and preserve Good is also deemed as unworthy because seeing Evil as separated from Good is too small-minded, and cannot do the job of preserving life.
The other avengers could not lift the hammer because they all have the intention to use the hammer to rid Evil and preserve Good, which is far from the hammer's purpose to preserve life.
Vision was also heard saying that he is on the side of life, to protect it. He has the intention to preserve life without being distracted by the notions of judging things to be simply Good or Evil. This is why Vision was deemed worthy to use the hammer.
The other avengers could not hold the hammer because they perceive the small picture rather than the bigger picture to preserve and protect life:
1) Black Widow is an assassin whose main job is to kill,
2) Tony is materialistic and thoughtless and does not care much about preserving other life but his friends and his own. He has a history of selling weapons to clients that got civilians killed by his weapons.
3) Captain America is a soldier who's job is to kill.
4) Warhammer is a soldier who kills
5) Hawkeye is also a soldier who kills
Even though Captain America can move the hammer a little because he is leader of the Avengers and can see the bigger picture compared to the other members(besides Thor), the Captain still cannot see the bigger picture in the same level like Thor and Vision.
Therefore, the only blindingly obvious reason how a person is deemed to be worthy enough to lift the hammer is if he/she would use it properly with the bigger intention to preserve life.
The hammer is only meant to preserve and protect life, it would not be usable for those who want to use it without holding this purpose in mind.