“Accessibility by design” is a vital idea for Microsoft, and one which underpins a lot of its synthetic intelligence-powered merchandise, together with Seeing AI.
Announced on Wednesday amongst a sequence of different AI instruments, Seeing AI is a free cellular utility designed to badist folks with visible impairments by narrating the world round them. The app — which is an ongoing badysis mission bringing collectively deep studying and Microsoft Cognitive Services — can learn paperwork, making sense of structural components resembling headings, paragraphs, and lists, in addition to establish a product utilizing its barcode.
It can moreover recognise and describe photos in different apps, and even pinpoint folks’s faces and supply an outline of their look, although digital camera high quality and lighting would possibly affect its description.
At the Microsoft Future of Artificial Intelligence occasion in Sydney, Kenny Johar Singh, a Melbourne-based cloud options architect at Microsoft, demonstrated Seeing AI, which he makes use of to badist navigate the bodily world.
Although he misplaced 75 p.c of his imaginative and prescient resulting from a degenerative retinal situation, expertise has been an “empowering force” that compelled him to pursue a profession within the trade, Singh stated.
Where earlier than Singh was reliant on his spouse to bridge the data hole between him and the bodily world, through the use of Seeing AI he is ready to be extra unbiased.
In entrance of friends, Singh used the app to scan a product, which the app accurately recognized as Bounce’s coconut lemon-flavoured pure vitality ball.
“What it’s actually done for me is that, now that it has detected the ball, I have the calorie calibration — proteins, carbohydrates, fats, and so forth. So I know what I’m picking and using which basically means that I can be totally independent now and my wife absolutely loves it because she doesn’t need to be dragged into stuff like this,” he advised friends on the Microsoft Summit.
Singh additionally pointed his telephone’s digital camera at Jenny Lay-Flurrie, Microsoft’s chief accessibility officer, and the Seeing AI app described Lay-Flurrie as a “54-year-old woman with brown hair wearing glbades looking happy”. The final three descriptive items of knowledge had been correct, although it was off-the-mark about her age.
“People who are blind are not the best at taking pictures, and we often get a lot of edge cases coming through with those pictures. Pictures that aren’t in the middle bracket of the high quality pictures that you can use to … adapt your algorithms and get some machine learning. They are in a lot of ways sometimes dirty pictures, but the overall quality of cognitive services will increase exponentially because you’re including this data sample,” Lay-Flurrie advised friends on the Microsoft Summit.
Microsoft’s newest accessibility options in platforms resembling Office 365 and Windows 10 are being utilized by the Australian National University’s legislation lecturer Cameron Roles, who stated in an announcement, “Now is definitely, in my view, the most exciting time in history to be blind.”
One of triplets, Roles, who can be a director with Vision Australia, was born three months early and the oxygen that saved his life left him blind. Of explicit use to him are the most recent accessibility options in platforms resembling Office 365 and Windows 10.
For instance, Microsoft has lately built-in its various textual content engine into the core of Windows 10, which Lay-Flurrie stated was a “serious step”. This means visually impaired customers that use third-party display readers or Microsoft’s personal Narrator, which reads textual content aloud and describes occasions, will be capable of get an outline of the contents in a picture, slightly than merely understanding there’s a picture on display.
The firm additionally lately launched Eye Control, a built-in eye monitoring function for Windows 10, enabling folks dwelling with motor neurone illness (MND) and different mobility impairments to navigate their computer systems. The function at the moment solely works with Swedish eye-tracking vendor Tobii’s Eye Tracker 4C, although Microsoft is working so as to add badist for different related gadgets.
Within Eye Control is a functionality referred to as “shape writing”, aimed toward dashing up typing by permitting the consumer to take a look at the primary and final letters of a phrase and “simply glancing at letters in between”. Microsoft stated in August “hint of the word predicted will appear on the last key of the word”, and if the prediction is wrong, the consumer can choose different predicted alternate options.
The firm has moreover launched colour-blindness filters in Windows 10, a situation that Lay-Flurrie stated is extra widespread that individuals badume, affecting one in 9 folks.
Lay-Flurrie, who has a listening to impairment, stated that whereas we have to think about the potential implications of AI in areas resembling privateness and safety, we also needs to take a look at the positives. She stated there are 1 billion folks dwelling with disabilities globally, and AI can empower these folks each of their day-to-day lives and in office environments.
“I also look at my daughter, who has autism, she’s 10. With autism, you don’t always understand social cues and social language, and facial expressions are not obvious to her what they mean. She often misinterprets what we’re saying … So I love the potential and the power and beginning to see a wave of innovation in the area of cognitive and mental health where you’re understanding those social cues, you’re using that visual stimulus to give examples, you’re helping to prompt what would be the next step to your learning through real life as opposed to sitting there … some of the therapeutic applications [require you] to sit there and watch YouTube videos,” she stated.
“I think there’s real-time applications that can change the lives and include in the same way you do in the workplace with PowerPoint Designer and position people as the geniuses that they are … And you need to be able to perform at the same level as anyone else. Cognitive services and some of these beautiful engines could give us that capability.”
Lay-Flurrie additionally stated folks with disabilities can lead or contribute considerably to innovation.
“People with disabilities have a unique lens on the world that could really give a mbadive input of innovation here and accelerate our path with AI,” she stated on the Microsoft Summit.
Microsoft’s chief storyteller Steve Clayton communicated an badogous sentiment on the occasion, saying improvements designed by and for folks dwelling with disabilities can show to be helpful extra broadly.
“When I started to learn about inclusivity in design, the dropped kerb was originally invented for people who are in wheelchairs. It turns out that the dropped kerb on a sidewalk is also incredibly useful if you’re carrying groceries or if you’re on a skateboard. There are these serendipitous moments I think we’ve found where we said, ‘Hey, we’re going to create a piece of technology that is for people with visual impairment or other disabilities’ that actually turned out to be incredibly useful for the rest of the world,” he stated.
Microsoft has been capable of combine AI into its merchandise, whereas providing new AI-powered merchandise, due to advances in pc imaginative and prescient, speech recognition, and pure language understanding.
The firm has developed applied sciences that may recognise speech with an error fee of 5.1 p.c and establish photos with an error fee of three.5 p.c.
Microsoft can be at the moment main a contest run by Stanford University that makes use of info from Wikipedia to check how nicely AI methods can reply questions on textual content pbadages. The competitors is predicted to generate outcomes that may be utilized in areas resembling Bing search and chatbot responses.
“This means that using AI’s deep learning, computers can recognise words in a conversation on par with a person, deliver relevant answers to very specific questions, and provide real-time translation,” the corporate stated in an announcement on Thursday.
“It also means that computers on a factory floor can distinguish between a fabricated part and a human arm, or that an autonomous vehicle can tell the difference between a bouncing ball and a toddler skipping across a street.”
In Australia, the University of Canberra has developed the Lucy and Bruce chatbots to streamline badist providers for college students and workers utilizing Microsoft Bot Framework and Microsoft Cognitive Services Language Understanding Intelligent Service.
Once launched, Lucy will hook up with the college’s Dynamics 365 platform, permitting college students to boost tickets when Lucy cannot discover the reply. The college can be exploring potentialities to make use of Bruce to permit IT service tickets to be logged by workers.
Australian Securities Exchange-listed packaging producer Pact Group has additionally labored with Microsoft utilizing its Cognitive Services Computer Vision for facial and objection recognition to spice up office security.
Pact’s Workroom Kiosk Demo can recognise particular person workers in a workshop atmosphere, detecting if they’re sporting acceptable security gear and monitoring their behaviour based mostly on an understanding of the duties particular person workers are authorised to carry out. Team leaders are mechanically alerted if there are potential points, and an on-site trial of the system shall be launched quickly.
Microsoft has additionally introduced developments to Translator, with expanded use of neural networks to enhance each textual content and speech translations in all of Translator’s supported merchandise.
For folks studying Chinese, the corporate will “soon” launch a brand new cellular utility from Microsoft Research Asia that may act as an all the time out there, AI-based language studying badistant.
The firm has moreover introduced Visual Studio Tools for AI for AI builders and information scientists, which it stated combines Visual Studio’s capabilities resembling debugging and wealthy enhancing, with the badist of deep studying frameworks resembling Microsoft Cognitive Toolkit, Google Tensorflow, and Caffe. Visual Studio Tools for AI leverages current code badist for Python, C/C++/C#, and provides extra badist for Cognitive Toolkit BrainScript, Microsoft stated.
AI capabilities for Azure IoT Edge — which allow builders to construct and take a look at container-based modules utilizing C, Java, .NET, Node.js and Python, and simplify the deployment and administration of workloads and machine studying fashions on the edge — are additionally now typically out there, the corporate stated.
“AI is about amplifying human ingenuity through intelligent technology that will reason with, understand, and interact with people and, together with people, help us solve some of society’s most fundamental challenges,” Clayton stated in an announcement.
Microsoft to combine Visual Studio with AI providers
Another piece of Microsoft’s ‘Open Mind Studio’ falls into place: A brand new extension enabling builders to make use of AI providers from inside Visual Studio.
Microsoft’s Visual Studio Live Share to enhance developer collaboration
Microsoft plans to debut a brand new developer collaboration service in early 2018 that would make it simpler for builders to work in tandem even when positioned remotely.
Microsoft will get data-fabulous at NYC occasion
Microsoft broadcasts Azure Databricks service, new Cosmos DB options, enterprise AI capabilities and extra at its annual Connect(); occasion in New York
Microsoft streamlines huge information badytics with new Azure providers (TechRepublic)
New database choices and providers in Microsoft’s Azure cloud platform had been unveiled as we speak at Microsoft’s Connect(); 2017 convention, alongside GitHub’s adoption of GVFS.
Microsoft reveals Azure IoT Edge: Putting AI on the furthest reaches of your community (TechRepublic)
Redmond revealed quite a lot of new providers to badist corporations reap the benefits of AI, starting from new badytics instruments to simpler methods to include machine studying into software program.