Technology

Accessibility has long played an integral role at AppleWorldwide Developers Conference, and it certainly had its presence felt in 2018. There are some constant elements of the week, such as the labs, sessions, and mid-week social that brings together members of Apple Accessibility group, developers and others interested in the disabled community to celebrate inclusiveness and accessible design.

This year had a different vibe. After attending the keynote and speaking with numerous people at Apple during the week, one feeling that has resonated with me is that accessibility, conceptually, has become a mandatory part of not only how Apple designs its products, but of the Apple ecosystem at large. To be mindful of accessibility is now, more than ever, an expectation.

Several student scholarship winners I spoke to eagerly expressed their desire to learn more about what accessibility is, how it works and how to best incorporate it into their apps. They truly want to build tools for everyone.

The announcements made this year were less about discrete features for accessibilitysake and more about how the new features, as Apple designed them, are inherently accessible. The Apple Design Awards, for instance, has included an accessibility category the last few years, but not so this year. In fact, accessibility has always been part of the criteria for selecting winners, and Apple says this yearwinners have support for accessibility built in. Calzy, a calculator app from indie developer Raja Vijayaraman, supports Dynamic Type.

Accessible announcements

As a disabled reporter who has covered the last several Apple media events, I&ve attended enough now that I&ve come to realize there is an interesting (to me) accessibility angle to how the company structures its events and how it supports attendees with disabilities. That is another story for another time, but I found myself thinking about it duringMondaykeynote address.

The slides Apple showed were the most readable I&ve seen yet.As someone who has low vision, this is critically important. The white, bold San Francisco typeface set against the black background made for such high contrast that I had no problems seeing every slide — even the word-cloud ones listing ancillary features. This made my job covering the news easier, because I wasn&t straining my eyes in order to get important information.

Accessibility-wise,Monday&skeynote slides were infinitely more visually friendly than the ones used at the March event in Chicago. As pretty and appropriately themed as they were, I found those slides difficult to see; I liked WWDCmuch better.

The lesson here is in how pervasive and dynamic accessibility can be. Accessibility is everywhere, and the relevance in this case is that it extends way beyond any new software features.

Accessibility for everyone

There are a handful of new mainstream features across Appleplatforms that the company feels has great potential in an accessibility context. This epitomizes the idea of accessibility for everyone—software not built expressly for accessibility, but designed in such a way that users of all abilities can benefit.

One such feature is Group FaceTime in iOS 12. Apple famously included deaf users in an iPhone 4 commercial many moons ago, and FaceTime has remained a popular method of communication for many users in the deaf community.

Whereas previously the feature was essentially a one-on-one conversation, the arrival of iOS 12 this fall will make it possible to converse with up to 32(!) people at once. For the deaf and hard-of-hearing, the ability to include oneentire family (or friends or co-workers) should make FaceTime an even more compelling technology for deaf users.

Another example is the Walkie-Talkie mode in watchOS 5. I&ve seen chatter on Twitter that it seems like a frivolous addition, but in actuality it can be practical.

Imagine you&re someone whoa caregiver for a person with severe physical impairments (or simply elderly) and both of you have an Apple Watch. With Walkie-Talkie, you can &radio& each other from separate locations in a home or care facility right from your wrists. No need to iMessage or make a phone call or ring a bedside call button. All you need is your Apple Watch and the Walkie-Talkie mode.

Lastly, Siri Shortcuts. While Applemarketing materials are pitching Siri Shortcuts in iOS 12 as a time-saving, convenient way to get things done, there also are key accessibility ramifications as well.

The ability to, for instance, order coffee at Starbucks or Philz without needing to remember to do it—or, crucially, how to do it in an app—can be streamlined with Siri Shortcuts. This has major implications for alleviating cognitive load and stress (in addition to being a time-saver), and has enormous potential to positively impact executive functionfor users who have certain cognitive delays.

But itmore than just cognition; the automation that Siri Shortcuts provides can also benefit those with limited fine-motor skills, who may struggle with the rigor of multiple taps and swipes.

In addition to the features above, there is a host of others whose accessibility promise excites Apple. The new dark mode in macOS Mojave, for example, boosts contrast considerably, which should help Mac users see better and guard against eye fatigue or screen glare.

AirPods with Live Listen

There was one notable hardware-specific feature not announced during the keynote. As Ireported this week, Live Listen — a feature previously only available to compatible Made for iPhone hearing aids — is coming to AirPods with the release of iOS 12.

The addition of Live Listen is noteworthy because it will allow someone with limited hearing to better hear speakers in noisy environments or from across a room. This functionality isn&t meant as a full replacement for a professional-grade hearing aid, but it certainly is a huge deal for the hard-of-hearing who want to use AirPods. Now they can use Live Listen to hear better without having to spend additional money on dedicated hardware.

Apple says Live Listen is included in the first developer beta of iOS 12, so anyone curious about it can test it out now.

Miscellany

Unlike past years, there aren&t any all-new discrete accessibility features across Appleplatforms this year. This follows with the theme that accessibility is interwoven into the banner features Apple is promoting in their marketing copy.

Nonetheless, there is a smattering of enhancements across Appleplatforms that are worth mentioning. Notable ones include the ability to use the Siri voice on iOS as the voice for Speak Selection, where you highlight a body of text and have Siri read it aloud. The Siri voice is now set as the default.

Another enhancement, pertaining to the Smart Annotations feature announced for iWork in March, is OCR support for handwritten notes, which will read them aloud. Blind and low vision users can now hear text markup in documents if they can&t see it.

Finally, the Touch Bar. VoiceOver users with Touch Bar MacBook Pros now have the ability to create custom automator scripts right from the Touch Bar with VoiceOver turned on.

The work Apple has put into making accessibility a focal point of the conference the last few years is bearing serious fruit this time around. Developers heard the message.

Tokyo-headquartered Rakuten, Japananswer to Amazon, is acquiring the Silicon Valley mobile ordering and pickup startup Curbside, the companies announced today. Terms of the all-cash deal were not disclosed, but Curbside had previously raised between $40 million and $50 million from investors including CVS, Index Ventures, Sutter Hull Ventures, AME Cloud Ventures, Qualcomm Ventures, Chicago Ventures, and others.

Founded in 2013 by former Apple engineers with backgrounds in location-based technology, Curbside was one of the early startups to capitalize on the idea that e-commerceexpansion won&t entirely involve ship-to-home deliveries, but could also include the convenience of online ordering with a curbside pickup option at bricks-and-mortar retailers to speed things up.

The company first launched its own mobile shopping app with a limited set of partners, including a shopping center in San Jose and a handful of San Francisco Bay area Target stores. The Target test wrapped up some time ago, and Target instead launched its own Drive Up curbside pickup this year & possibly encouraged by the potential it saw through its tests with the third party.

Curbside then went on to power mobile orders and store pickup for CVS, which invested in the company as a strategic partner back in 2016.

The startup also developed an SDK for mobile app developers called ARRIVE that allows retailers to see when customers are arriving at their location for things like order pickup, mobile order ahead, and appointment checkin. CVS, Sephora, Chipotle, Nordstrom, Pizza Hut, Chevron, Boston Market, Westfield, HEB, and Yelp are listed on the Curbside website as ARRIVE customers.

For strategic partner CVS in particular, Curbside is available at thousands of locations across the U.S. for order ahead and pickup.

ARRIVE has since become the primary business for Curbside, and now contributes to the majority of its revenue. Curbside CEO Jaron Waldman says half of the top ten QSR(quick serve restaurants) are now using ARRIVE, as the restaurant side of the ARRIVE business has really taken off. These operations will continue as planned, Curbside says.

Across its customer base, Curbside has powered several million curbside pickups to date, and has over 8,000 U.S. locations, as well as some traction in non-U.S. markets, including Canada and India (Pizza Hut).

As for Rakuten, the acquisition opens up a lot of opportunities in terms of connecting retailers and merchants with customers, particularly in order ahead and pickup.

&Therea shifthappening in consumer behavior where people want to save time. They want to order ahead on their mobile device and have things ready when they get there & whether thatin the store or curbside in front of the store,& explains Waldman. &ARRIVE works really well in both use cases…And we&re also helping [retailers] really measure the performance of the individual store,& he says. &The folks that have that physical world touch point needs to ensure that stores are performing the consumers are actually not waiting that long.&

Rakuten today reaches over 1 billion members worldwide, including those on its own Rakuten Ichiba online marketplace in Japan.

Those online merchants could potentially take advantage of the Curbside technology to offer the option of order pickup for their customers, alongside their existing delivery options.

These sorts of integrations may not be limited only to the marketplace itself, however. Rakuten also has other consumer touch points, like communications app Viber and eBates.com, where Curbside could also reach the consumer audience in various ways. And it has investments in companies like Pinterest, Lyft, Cabify, and others, where it could do the same. It also has a partnership with Walmart in Japan in online grocery, where Curbside pickup could get involved.

In fact, Waldman says there are at least half a dozen opportunities inside Rakuten it could now pursue. The question is really which ones will it go after first.

&I can say therea ton of interest to bring this technology to Japan,& he notes. &They already touch 100 million consumers in Japan & their registered users. We could hook up this really big ecosystem of consumers and merchants,& adds Waldman.

Healso excited to broaden the Curbside offering thanks to Rakutenexpanded resources.

&We don&t have an out of the box payment solution. Therea lotlot in the Rakuten ecosystem thatgoing to allow us to to broaden that offering, which means that we can add more value for for our merchants and our retail partners,& he says.

And as a part of Rakuten, Curbside can more quickly expand to global markets & something it had just begun doing with smaller launches in Canada and India.

&Rakuten was founded on the philosophy of empowering merchants to reach consumers in new ways. In 1997 it was selling on the internet. Today itmobile commerce. Curbside has a unique ability to surprise and delight consumers with fun and convenient ways to shop while empowering local merchants,& saysYaz Iida, President of Rakuten USA. &It fits into our unique philosophy and will be a piece of Rakutenecosystem of internet services.&

Curbside60-person team, for the time being, will remain in its Palo Alto offices, though it may later move to Rakutenoffices in San Mateo.Waldman remain as Rakuten Curbside CEO.

The startup had raised between $40 million and $50 million, including its undisclosed investment by CVS in 2016. Before that, it had raised $34.5 million, according to Crunchbase.

At AppleWWDC 2018— an event some said would be boring this year with its software-only focus and lack of new MacBooks and iPads — the company announced what may be its most important operating system update to date with the introduction of iOS 12. Through a series of Siri enhancements and features, Apple is turning its iPhone into a highly personalized device, powered by its Siri AI.

This &new AI iPhone& — which, to be clear, is your same ol& iPhone running a new mobile OS — will understand where you are, what you&re doing and what you need to know right then and there.

The question now is will users embrace the usefulness of Siriforthcoming smarts, or will they find its sudden insights creepy and invasive

Siri Suggestions

After the installation of iOS 12, SiriSuggestions will be everywhere.

In the same place on the iPhone Search screen where you today see those Siri suggested apps to launch, you&ll begin to see other things Siri thinks you may need to know, too.

For example, Siri may suggest that you:

• Call your grandma for her birthday.
• Tell someone you&re running late to the meeting via a text.
• Start your workout playlist because you&re at the gym.
• Turn your phone to Do Not Disturb at the movies.

And so on.

These will be useful in some cases, and perhaps annoying in others. (It would be great if you could swipe on the suggestions to further train the system to not show certain ones again. After all, not all your contacts deserve a birthday phone call.)

Siri Suggestions will also appear on the Lock Screen when it thinks it can help you perform an action of some kind. For example, placing your morning coffee order — something you regularly do around a particular time of day — or launching your preferred workout app, because you&ve arrived at the gym.

These suggestions even show up on Apple WatchSiri watch face screen.

Apple says the relevance of its suggestions will improve over time, based on how you engage.

If you don&t take an action by tapping on these items, they&ll move down on the watch facelist of suggestions, for instance.

AI-powered workflows

These improvements to Siri would have been enough for iOS 12, but Apple went even further.

The company also showed off a new app called Siri Shortcuts.

The app is based on technology Apple acquired from Workflow, a clever — if somewhat advanced — task automation app that allows iOS users to combine actions into routines that can be launched with just a tap. Now, thanks to the Siri Shortcuts app, those routines can be launched by voice.

Onstage at the developer event, the app was demoed by Kim Beverett from the Siri Shortcuts team, who showed off a &heading home& shortcut she had built.

When she tells Siri she&heading home,& her iPhone simultaneously launched directions for her commute in Apple Maps, set her home thermostat to 70 degrees, turned on her fan, messaged an ETA to her roommate and launched her favorite NPR station.

Thatarguably very cool — and it got a big cheer from the technically minded developer crowd — but itmost certainly a power user feature. Launching an app to build custom workflows is not something everyday iPhone users will do right off the bat — or in some cases, ever.

Developers to push users to Siri

But even if users hide away this new app in their Apple &junk& folder, or toggle off all the Siri Suggestions in Settings, they won&t be able to entirely escape Siripresence in iOS 12 and going forward.

Thatbecause Apple also launched new developer tools that will allow app creators to build directly into their own apps integrations with Siri.

Developers will update their apps& code so that every time a user takes a particular action — for example, placing their coffee order, streaming a favorite podcast, starting their evening jog with a running app or anything else — the app will let Siri know. Over time, Siri will learn users& routines — like, on many weekday mornings, around 8 to 8:30 AM, the user places a particular coffee order through a coffee shop apporder ahead system.

These will inform those Siri Suggestions that appear all over your iPhone, but developers will also be able to just directlyprod the user to add this routine to Siri right in their own apps.

In your favorite apps, you&ll start seeing an &Add to Siri& link or button in various places — like when you perform a particular action — such as looking for your keys in Tileapp, viewing travel plans in Kayak, ordering groceries with Instacart and so on.

Many people will probably tap this button out of curiosity — after all, most don&t watch and rewatch the WWDC keynote like the tech crowd does.

The &Add to Siri& screen will then pop up, offering a suggestion of voice prompt that can be used as your personalized phase for talking to Siri about this task.

In the coffee ordering example, you might be prompted to try the phrase &coffee time.& In the Kayak example, it could be &travel plans.&

You record this phrase with the big, red record button at the bottom of the screen. When finished, you have a custom Siri shortcut.

You don&t have to use the suggested phrase the developer has written. The screen explains you can make up your own phrase instead.

In addition to being able to &use& apps via Siri voice commands, Siri can also talk back after the initial request.

It can confirm your request has been acted upon — for example, Siri may respond, &OK. Ordering. Your coffee will be ready in 5 minutes,& after you said &Coffee time& or whatever your trigger phrase was.

Or it can tell you if something didn&t work — maybe the restaurant is out of a food item on the order you placed — and help you figure out what to do next (like continue your order in the iOS app).

It can even introduce some personality as it responds. In the demo, Tileapp jokes back that it hopes your missing keys aren&t &under a couch cushion.&

There are a number of things you could do beyond these limited examples — the App Store has more than 2 million apps whose developers can hook into Siri.

And you don&t have to ask Siri only on your phone — you can talk to Siri on your Apple Watch and HomePod, too.

Yes, this will all rely on developer adoption, but it seems Apple has figured out how to give developers a nudge.

Siri Suggestions are the new Notifications

You see, as Sirismart suggestions spin up, traditional notifications will wind down.

In iOS 12, Siri will take note of your behavior around notifications, and then push you toturn off those with which you don&t engage, or move them into a new silent mode Apple calls &Delivered Quietly.& This middle ground for notifications will allow apps to send their updates to theNotification Center, but not the Lock Screen. They also can&t buzz your phone or wrist.

At the same time, iOS 12new set of digital well-being features will hide notifications from users at particular times — like when you&ve enabled Do Not Disturb at Bedtime, for example. This mode will not allow notifications to display when you check your phone at night or first thing upon waking.

Combined, these changes will encourage more developers to adopt the Siri integrations, because they&ll be losing a touchpoint with their users as their ability to grab attention through notifications fades.

Machine learning in photos

AI will further infiltrate other parts of the iPhone, too, in iOS 12.

A new &For You& tab in the Photos app will prompt users to share photos taken with other people, thanks to facial recognition and machine learning. And those people, upon receiving your photos, will then be prompted to share their own back with you.

The tab will also pull out your best photos and feature them, and prompt you to try different lighting and photo effects. A smart search feature will make suggestions and allow you to pull up photos from specific places or events.

Smart or creepy

Overall, iOS 12AI-powered features will make Appledevices more personalized to you, but they could also rub some people the wrong way.

Maybe people won&t want their habits noticed by their iPhone, and will find Siri prompts annoying — or, at worst, creepy, because they don&t understand how Siri knows these things about them.

Apple is banking hard on the fact that itearned users& trust through its stance on data privacy over the years.

And while not everyone knows thatSiri is doesa lot of its processing on your device,not in the cloud, many do seem to understand that Apple doesn&t sell user data to advertisers to make money.

That could help sell this new &AI phone& concept to consumers, and pave the way for more advancements later on.

But on the flip side, if Siri Suggestions become overbearing or get things wrong too often, it could lead users to just switch them off entirely through iOS Settings. And with that, Applebig chance to dominate in the AI-powered device market, too.