A few months ago, I wrote a review for RSBC on the Lens App, a free OCR app from Microsoft that allows you to take a picture of text and have it read back.
Despite the positive feedback and how it offers a free alternative to paid OCR apps, it was still rather fiddley and cumbersome to read a document quickly.
I’m happy to say Microsoft has stepped up their game and launched a brand new app called Seeing AI, an app that can extract texts from images, recognise faces, describe pictures and more! Like the Microsoft Lens app, it is completely free.
You can download the app by searching for Seeing AI on the App Store.
Once you open the app for the first time it will ask to have access to your device’s s camera. Allow this as this app needs this to work.
You will then be presented with a tutorial that you can read through. After flicking through the pages, tap on get started and agree to their terms.
You are now on the main screen of the app.
The top left is a menu button where you can set settings and face recognition, more about that later.
A quick help button is on the top right that gives you more information on the current mode.
In the middle is a camber view of your back facing camera, a nice large area for you to tap to take a picture if the app doesn’t automatically do it for you.
Finally at the bottom of the screen you have your modes, call Channels. These are, Short Text, Document, Product, Person and Scene Beta. You can use VoiceOver gestures to flick through each channel.
As you enter the channel for the first time, the quick help page will pop up alongside a short video.
Perhaps the app’s most useful feature is the short text channel. It is great for reading a small amount of text quickly. Things such as names on envelopes, or dialog boxes on computer screens. Enter this channel and point your phone at some text and it will read instantly, without you having to take a picture and worry if you got it right or not.
During testing I’ve found this channel extremely useful when my computer stopped talking and I needed to see what was on the screen to figure out why.
This channel is for larger documents such as letters or printed work sheets. Just like the first channel, point your phone at a piece of paper, and it will give you audio guidance on how to position the paper under the phone. A handy tip is to place the phone in the middle of the paper and slowly bring it up so it’s parallel. As it gets further away, more of your paper will come into the camera view, so don’t be too shy to go higher and stand up. When you get it right, it will tell you to hold your phone steady and it will automatically snap a picture of the text, and then read it with VoiceOver.
Unfortunately I didn’t’ manage to get this working, (probably due to the service being new in the UK), so it didn’t recognise any of the barcodes I showed it. However from watching the main demo, it will identify a barcode on a product by audio sounds, and once it’s in focus, automatically scans it. After processing it will tell you the type of product and a more information button will tell you more about the item.
This is great to take pictures of people and their faces, the app will tell you where they are positioned so you can get them in the whole frame. It will even try to guess the age of the person, hair colour and have a guess at their emotions.
Warning: do not get offended if they get your age wrong. It can be out by over 10 years sometimes.
Remember I mentioned the menu button on the top left of the main screen? Double tap on this and go to Face Recognition.
Here you can teach the app whose face is who. It will always start off with the front facing camera, so if you are taking a picture of someone else and not yourself switch it back to the back facing one.
You need to take three pictures of the person, so it helps it learn, after all three, it will ask you to name the person. Once you entered a name and tap done, go back to the main screen and on the Person channel, if you point your phone at a face that you have saved, it will tell you who’s in front of you!
Continue to add more people to your list so when you are using your phone to see who’s around you, they will be announced. ?Could be handy when walking into a busy meeting for checking out who’s around the table.
This is a very basic image identification feature. Take a picture of what’s in front of you and it will describe what’s around you as best as it can. For instance it might say “An office with desks and computers”
Describing pictures from other appss
Another handy feature is Seeing AI’s ability to describe pictures shared from other apps, like Twitter and Whatsapp.
On Twitter find a tweet with an image. Double tap on the tweet, scroll down to where it says image, and then double tap and hold. The share sheet will come up. Select more, and then swich Seeing AI on.
Hit done and then double tap on ‘Recognise with Seeing AI’ which should be at the bottom of the sharing options next to the more button.
Then wait and Seeing AI will describe the image.
It does require an active internet connection to use, so if you don’t have WiFi or cellular you won’t be able to use the features.
All in all, I am really impress with the results of the app, and things like the real-time text to speech makes it a must have, and even better it is completely free!
The speed of results is outstanding recognising both faces and text, saving time.
Try it out and let me know what you think!