Microsoft just released an incredible new app that helps blind people see the world around them (MSFT)


Artificial intelligence is helping people do extraordinary things. At Microsoft, it’s helping blind people see the world around them like never before.

In March of last year, Microsoft showed off a prototype of its Seeing AI app, which looked very promising at the time, but the company released the free app to all iOS users on Wednesday.

The Seeing AI app uses your smartphone’s camera to identify things in your environment — people, objects, and even emotions — to provide important context for what’s going on around you.

Take a look.

Meet Saqib Shaikh.

Shaikh lost the use of his eyes when he was just seven years old.

Shaikh lost the use of his eyes when he was just seven years old.Shaikh lost the use of his eyes when he was just seven years old.

 

Shortly after, Shaikh was introduced to talking computers at a school for the blind. This inspired him to become a programmer.

Shortly after, Shaikh was introduced to talking computers at a school for the blind. This inspired him to become a programmer.Shortly after, Shaikh was introduced to talking computers at a school for the blind. This inspired him to become a programmer.

Shaikh has been working at Microsoft for the last 10 years as a software engineer.

Shaikh has been working at Microsoft for the last 10 years as a software engineer.Shaikh has been working at Microsoft for the last 10 years as a software engineer.

He’s been working on an idea he’s had since his college days: an app that can tell you at any moment what’s going on around you.

He's been working on an idea he's had since his college days: an app that can tell you at any moment what's going on around you.He’s been working on an idea he’s had since his college days: an app that can tell you at any moment what’s going on around you.

Shaikh teamed up with like-minded engineers to work on this very app using Microsoft’s intelligence APIs.

Shaikh teamed up with like-minded engineers to work on this very app using Microsoft's intelligence APIs.Shaikh teamed up with like-minded engineers to work on this very app using Microsoft’s intelligence APIs.

 

The app, called Seeing AI, works on smartphones (just iOS for now), but last year’s demo also showed the app working with Pivothead’s SMART glasses, pictured here. Shaikh can simply swipe forward on the glasses and they’ll tell him what he’s looking at.

The app, called Seeing AI, works on smartphones (just iOS for now), but last year's demo also showed the app working with Pivothead's SMART glasses, pictured here. Shaikh can simply swipe forward on the glasses and they'll tell him what he's looking at.The app, called Seeing AI, works on smartphones (just iOS for now), but last year’s demo also showed the app working with Pivothead’s SMART glasses, pictured here. Shaikh can simply swipe forward on the glasses and they’ll tell him what he’s looking at.

 

“I think it’s a man jumping through the air doing a trick on a skateboard,” Microsoft’s AI says aloud.

"I think it's a man jumping through the air doing a trick on a skateboard," Microsoft's AI says aloud.“I think it’s a man jumping through the air doing a trick on a skateboard,” Microsoft’s AI says aloud.

 

The Seeing AI app has other applications, such as working in groups.

The Seeing AI app has other applications, such as working in groups.The Seeing AI app has other applications, such as working in groups.

 

The Seeing AI app can tell you whether or not people are really listening to what you’re saying, and it can describe the general age and gender of the people around you, as well as their emotions.

The Seeing AI app can tell you whether or not people are really listening to what you're saying, and it can describe the general age and gender of the people around you, as well as their emotions.The Seeing AI app can tell you whether or not people are really listening to what you’re saying, and it can describe the general age and gender of the people around you, as well as their emotions.

 

 

“I see two faces: 40-year-old man with a beard looking surprised; 20-year-old woman looking happy.”

 

The app is also good at reading out text.

The app is also good at reading out text.The app is also good at reading out text.

 

 

You can use the app to take a picture of the text, and the app will actually guide you to ensure you’re capturing the full text of what you’re looking at.

You can use the app to take a picture of the text, and the app will actually guide you to ensure you're capturing the full text of what you're looking at.You can use the app to take a picture of the text, and the app will actually guide you to ensure you’re capturing the full text of what you’re looking at.

 

“Move camera to the bottom right and away from the document,” Microsoft’s AI says.

"Move camera to the bottom right and away from the document," Microsoft's AI says.“Move camera to the bottom right and away from the document,” Microsoft’s AI says.

 

 

Once you take the picture, Microsoft’s AI recognizes the text. It can even identify and read headings in case you don’t want to know about every single item on a restaurant’s menu, for example.

Once you take the picture, Microsoft's AI recognizes the text. It can even identify and read headings in case you don't want to know about every single item on a restaurant's menu, for example.Once you take the picture, Microsoft’s AI recognizes the text. It can even identify and read headings in case you don’t want to know about every single item on a restaurant’s menu, for example.

 

 

“Years ago, this was science fiction,” Shaikh says. “I never thought it would be something you could actually do, but artificial intelligence is improving at an ever faster rate and I’m really excited to see where we can take this.”

"Years ago, this was science fiction," Shaikh says. "I never thought it would be something you could actually do, but artificial intelligence is improving at an ever faster rate and I'm really excited to see where we can take this."“Years ago, this was science fiction,” Shaikh says. “I never thought it would be something you could actually do, but artificial intelligence is improving at an ever faster rate and I’m really excited to see where we can take this.”

You can check out the full video of Microsoft’s Seeing AI app here.

If you have an iPhone, you can download the free Seeing AI app right here.

Previous 4 ways to boost your Wi-Fi speed
Next The Right Way to Start a Law Blog?

No Comment

Leave a reply

Your email address will not be published. Required fields are marked *