This week in New York, Microsoft revealed its new Copilot features focused on consumers. On stage, Microsoft AI CEO, Mustafa Suleyman spoke to the role AI is playing in redefining our everyday relationship with computers, the need for Copilot to be personable and relatable and the obligation of balancing innovation and consumers’ acceptance. Both on stage and in a penned memo Suleyman explained that we are currently witnessing an extraordinary technological paradigm shift. In just a few years, our computers have learned to speak our languages, see what we see, and hear what we hear. This remarkable evolution of technology raises a crucial point: while the capabilities of AI are impressive, it is the impact on human lives that truly matters. Technology must serve humanity, enriching lives, creating opportunities, and fostering connections.

A highlight of this extensive update is Copilot Vision, a feature allowing AI to actually “see” and interpret what users are viewing on their screens. This means Copilot can provide immediate assistance or suggestions based on visual context, enhancing user productivity by eliminating the need for manual input or searches for information related to on-screen content. The demo shown in stage showed a user asking about what movie they should watch based on the genre and mood they were in and while they had their browser on Rotten Tomatoes.

While some have expressed concerns about privacy, I am convinced that this feature offers a secure introduction for users to familiarize themselves with Copilot’s capabilities using public information, from a limited list of pre-proved sites, before delving into more sensitive matters. Crucially, it is important to emphasize that Copilot’s assistance is always user-initiated and that no query is ever stored. This ensures peace of mind, allowing users to confidently explore the potential of this advanced AI tool without compromising their privacy.

Copilot Voice represents a groundbreaking advancement, poised to revolutionize user interactions into seamless, natural dialogues. This state-of-the-art voice technology has been refined to feature human-like cadence, enabling it to gracefully handle interruptions and pauses, much like a genuine conversational partner would. This creates a more engaging and intuitive interaction that Microsoft envisions will make technology feel more approachable and less mechanical.

For instance, when I consulted Copilot about the pros and cons of owning a sausage dog, I received a well-rounded answer. During the response, I was able to interrupt to inquire about the preference for long or short hair varieties. Impressively, when I spoke over Copilot’s answer, it paused momentarily to assimilate my question before continuing, akin to how a human might pause to gather their thoughts if interrupted mid-sentence. The interaction felt remarkably natural, characterized by an appropriate tone, a touch of humor, and impressive accuracy and knowledge.

This enhancement in Copilot Voice transforms AI from a mere tool into an engaging digital companion, redefining our relationship with technology by truly reflecting the nuances of human conversation.

As Suleyman shared insights from user feedback, it became strikingly apparent that individuals are at vastly different stages in integrating AI into their daily lives. For some, AI like Copilot has become a trusted confidant, ready to provide insights on deeply personal queries. For others, it remains a transactional tool for basic tasks. This dichotomy presents both a significant risk and an exciting opportunity for Microsoft. The risk lies in advancing too rapidly towards crafting an overly empathetic AI companion that could ultimately disappoint. However, the opportunity is monumental: to forge an enduring, personal bond with an AI agent that users find indispensable—an agent they never imagine leaving behind.

Much like the Vision feature, Copilot’s new Think Deeper capability is designed to enhance Microsoft’s agent by making it more adaptable. Think Deeper equips Copilot with the ability to tackle more intricate problems, leveraging “reasoning models” that allow for thoughtful, step-by-step responses, according to Microsoft. Think Deeper seems to be utilizing a customized version of OpenAI’s o1 model is accessible to a select number of users within the Copilot Labs community in Australia, Canada, New Zealand, the U.S., and the U.K.

In recent years, Microsoft has been eager to extend its reach and capture consumer engagement beyond the realm of PC productivity and gaming. I strongly believe that Copilot could be the way to do it.

The new Copilot will be available on iOS and Android through a dedicated app. Although Microsoft’s lack of integration within operating systems has traditionally restricted its ability to engage users beyond Microsoft Edge, this app-based approach could enhance visibility and highlight Copilot’s value. With discovery currently being a challenge for both Gemini and Apple Intelligence, presenting Copilot as an accessible application creates an opportunity to bridge the gap between smartphones and PCs. By seamlessly delivering value across these platforms, Microsoft has the chance to make Copilot compelling enough for users to consider it over the built-in options on their smartphones.

For Microsoft’s Copilot to see widespread adoption, striking the perfect balance is essential. The company must meet consumers where they are now, while also inspiring them to explore and embrace new capabilities, enriching their interaction with their AI companion. By maintaining transparency and accountability throughout this process, Microsoft can cultivate the trust necessary to guide consumers on an exciting journey of discovery. As users grow more confident in these interactions, they’ll be more inclined to delve further, unlocking the full potential of their digital companion and transforming how they engage with technology.

Join the newsletter and stay up to date