Technology

DeepMind foray into digital health services continues to raise concerns.The latest worries are voiced by a panel of external reviewers appointed by the Google-owned AI company to report on its operations after its initial data-sharing arrangements with the U.K.National Health Service (NHS) ran into a major public controversy in 2016.

The DeepMind Health Independent Reviewers&2018 reportflags a series of risks and concerns, as they see it, including the potential for DeepMind Health to be able to &exert excessive monopoly power& as a result of the data access and streaming infrastructure thatbundled with provision of the Streams app — andwhich, contractually, positions DeepMind as the access-controlling intermediary between the structured health data and any other third parties that might, in the future, want to offer their own digital assistance solutions to the Trust.

While the underlying FHIR (aka, fast healthcare interoperability resource) deployed by DeepMind for Streams uses an open API, the contract between the company and the Royal Free Trust funnels connections via DeepMindown servers, and prohibits connections to other FHIR servers. A commercial structure that seemingly works against the openness and interoperability DeepMindco-founder Mustafa Suleyman has claimed to support.

&There are many examples in the IT arena where companies lock their customers into systems that are difficult to change or replace. Such arrangements are not in the interests ofthe public. And we do not want to see DeepMind Health putting itself in a position where clients, such as hospitals, find themselves forced to stay with DeepMind Health even if it is no longer financially or clinically sensible to do so;we want DeepMind Health to compete on quality and price, not by entrenching legacy position,& the reviewers write.

Though they point toDeepMind&stated commitmentto interoperability of systems,& and &their adoption of the FHIR open API& as positive indications, writing: &This means that there is potential for many other SMEs to become involved, creating a diverse and innovative marketplace which works to the benefit of consumers, innovation and the economy.&

&We also note DeepMind Healthintention to implement many of the features of Streams as modules which could be easily swapped, meaning that they will have to rely on being the best to stay in business,& they add.

However, stated intentions and future potentials are clearly not the same as on-the-ground reality. And, as it stands, a technically interoperable app-delivery infrastructure is being encumbered by prohibitive clauses in a commercial contract — and by a lack of regulatory pushback against such behavior.

The reviewers also raise concerns about an ongoing lack of clarity around DeepMind Healthbusiness model — writing: &Given the current environment, and with no clarity about DeepMind Healthbusiness model, people arelikely to suspect that there must be an undisclosed profit motive or a hidden agenda. We do not believe this to be the case, but would urge DeepMind Health to be transparent about their business model, and their ability to stick to that without being overridden by Alphabet. For once an ideaof hidden agendas is fixed in peoplemind, it is hard to shift, no matter how much a company is motivated by the public good.&

&We have had detailed conversations about DeepMind Healthevolving thoughts in this area, and are aware that some of these questions have not yet been finalised. However, we would urge DeepMind Health to set out publicly what they are proposing,& they add.

DeepMind has suggested it wants to build healthcare AIs that are capable of charging by results. But Streams does not involve any AI. The serviceis also being provided to NHS Trusts for free, at least for the first five years — raising the question of how exactly the Google-owned company intends to recoup its investment.

Google of course monetizes a large suite of free-at-the-point-of-use consumer products — such as the Android mobile operating system; its cloud email service Gmail; and the YouTube video sharing platform, to name three — by harvesting peoplepersonal data and using that information to inform its ad targeting platforms.

Hence the reviewers& recommendation for DeepMind to set out its thinking on its business model to avoid its intentions vis-a-vis peoplemedical data being viewed with suspicion.

The companyhistorical modus operandi also underlines the potential monopoly risks if DeepMind is allowed to carve out a dominant platform position in digital healthcare provision — given how effectively its parent has been able to turn a free-for-OEMs mobile OS (Android) into global smartphone market OS dominance, for example.

So, while DeepMind only has a handful of contracts with NHS Trusts for the Streams app and delivery infrastructure at this stage, the reviewers& concerns over the risk ofthe company gaining &excessive monopoly power&do not seem overblown.

They are also worried about DeepMindongoing vagueness about how exactly it works with its parent Alphabet, and what data could ever be transferred to the ad giant — an inevitably queasy combination when stacked against DeepMindhandling of peoplemedical records.

The Google-owned U.K. AI company began its push into digital healthcare services in 2015, quietly signing an information-sharing arrangement with a London-based NHS Trust that gave it access to around 1.6 million peoplemedical records for developing an alerts app for a condition called Acute Kidney Injury.

It also inked an MoU with the Trust where the pair set out their ambition to apply AI to NHS data sets. (They even went so far as to get ethical signs-off for an AI project — but have consistently claimed the Royal Free data was not fed to any AIs.)

However, the data-sharing collaboration ran into trouble in May 2016 when the scope of patient data being shared by the Royal Free with DeepMind was revealed (via investigative journalism, rather than by disclosures from the Trust or DeepMind).

None of the ~1.6 million people whose non-anonymized medical records had been passed to the Google-owned company had been informed or asked for their consent. And questions were raised about the legal basis for the data-sharing arrangement.

Last summerthe U.K.privacy regulator concluded an investigation of the project — finding that the Royal Free NHS Trust had broken data protection rules during the appdevelopment.

Yet despite ethical questions and regulatory disquiet about the legality of the data sharing, the Streams project steamrollered on. And the Royal Free Trust went on to implement the app for use by clinicians in its hospitals, while DeepMind has also signed several additional contracts to deploy Streams to other NHS Trusts.

More recently, the law firm Linklaters completed an audit of the Royal Free Streams project, after being commissioned by the Trust as part of its settlement with the ICO. Though this audit only examined the current functioning of Streams. (There has been no historical audit of the lawfulness of peoplemedical records being shared during the build and test phase of the project.)

Linklaters did recommend the Royal Free terminates its wider MoU with DeepMind — and the Trust has confirmed to us that it will be following the firmadvice.

&The audit recommends we terminate the historic memorandum of understanding with DeepMind which was signed in January 2016. The MOU is no longer relevant to the partnership and we are in the process of terminating it,& a Royal Free spokesperson told us.

So DeepMind, probably the worldmost famous AI company, is in the curious position of being involved in providing digital healthcare services to U.K. hospitals that don&t actually involve any AI at all. (Though it does have some ongoing AI research projects with NHS Trusts too.)

In mid 2016, at the height of the Royal Free DeepMind data scandal — and in a bid to foster greater public trust — the company appointed the panel of external reviewers who have now produced their second report looking at how the division is operating.

And itfair to say that much has happened in the tech industry since the panel was appointed to further undermine public trust in tech platforms and algorithmic promises— including the ICOfinding that the initial data-sharing arrangement between the Royal Free and DeepMind broke U.K. privacy laws.

The eight members of the panel for the 2018 report are:Martin Bromiley OBE; Elisabeth Buggins CBE; Eileen Burbidge MBE; Richard Horton;Dr. Julian Huppert;Professor Donal O&Donoghue; Matthew Taylor; andProfessor Sir John Tooke.

In their latest report the external reviewers warn that the publicview of tech giants has &shifted substantially& versus where it was even a year ago — asserting that &issues of privacy in a digital age are if anything, of greater concern.&

At the same time politicians are also gazing rather more critically on the works and social impacts of tech giants.

Although the U.K. government has also been keen to position itself as a supporter of AI, providing public funds for the sector and, in itsIndustrial Strategy white paper, identifying AI and data as one of four so-called &Grand Challenges& where it believes the U.K. can &lead the world for years to come& — including specificallyname-checking DeepMind as one of a handful of leading-edge homegrown AI businesses for the country to be proud of.

Still, questions over how to manage and regulate public sector data and AI deployments — especially in highly sensitive areas such as healthcare — remain to be clearly addressed by the government.

Meanwhile, the encroaching ingress of digital technologies into the healthcare space — even when the techs don&t even involve any AI — are already presenting major challenges by putting pressure on existing information governance rules and structures, and raising the specter of monopolistic risk.

Asked whether it offers any guidance to NHS Trusts around digital assistance for clinicians, including specifically whether it requires multiple options be offered by different providers, the NHS& digital services provider, NHS Digital, referred our question on to the Department of Health (DoH), saying ita matter of health policy.

The DoH in turn referred the question to NHS England, the executive non-departmental body which commissions contracts and sets priorities and directions for the health service in England.

And at the time of writing, we&re still waiting for a response from the steering body.

Ultimately it looks like it will be up to the health service to put in place a clear and robust structure for AI and digital decision services that fosters competition by design by baking in a requirement for Trusts to support multiple independent options when procuring apps and services.

Without that important check and balance, the risk is that platform dynamics will quickly dominate and control the emergent digital health assistance space — just as big tech has dominated consumer tech.

But publicly funded healthcare decisionsand data setsshould not simply be handed to the single market-dominating entity thatwilling and able to burn the most resource to own the space.

Nor should government stand by and do nothing when therea clear risk that a vital area of digital innovation is at risk of being closed down by a tech giant muscling in and positioning itself as a gatekeeper before others have had a chance to show what their ideas are made of, and before even a market has had the chance to form.

If you&re already resentful of online dating culture and how it turned finding companionship into a game, you may not be quite ready for this: Crown, a new dating app that actually turns getting matches into a game. Crown is the latest project to launch from Match Group, the operator of a number of dating sites and apps including Match, Tinder, Plenty of Fish, OK Cupid, and others.

The app was thought up by Match Product Manager Patricia Parker, who understands first-hand both the challenges and the benefits of online dating & Parker met her husband online, so has direct experience in the world of online dating.

Crown won Match Groupinternal &ideathon,& and was then developed in-house by a team of millennial women, with a goal of serving womenneeds in particular.

The main problem Crown is trying to solve is the cognitive overload of using dating apps. As Match Group scientific advisor Dr. Helen Fisher explained a few years ago toWired, dating apps can become addictive because thereso much choice.

&The more you look and look for a partner the more likely it is that you&ll end up with nobody…Itcalled cognitive overload,& she had said. &There is a natural human predisposition to keep looking—to find something better. And with so many alternatives and opportunities for better mates in the online world, iteasy to get into an addictive mode.&

Millennials are also prone to swipe fatigue, as they spend an average of 10 hours per week in dating apps, and are being warned to cut down or face burnout.

Crownapproach to these issues is to turn getting matches into a game of sorts.

While other dating apps present you with an endless stream of people to pick from, Crown offers a more limited selection.

Every day at noon, you&re presented with 16 curated matches, picked by some mysterious algorithm. You move through the matches by choosing who you like more between two people at a time.

That is, the screen displays two photos instead of one, and you &crown& your winner. (Get it) This process then repeats with two people shown at a time, until you reach your &Final Four.&

Those winners are then given the opportunity to chat with you, or they can choose to pass.

In addition to your own winners, you may also &win& the crown among other brackets, which gives you more matches to contend with.

Of course, getting dubbed a winner is a stronger signal on Crown than on an app like Tinder, where itmore common for matches to not start conversations. This could encourage Crown users to chat, given they know theremore of a genuine interest since they &beat out& several others. But on the flip side, getting passed on Crown is going to be a lot more of an obvious &no,& which could be discouraging.

&Itlike a ‘Bachelorette&-style process of elimination that helps users choose between quality over quantity,& explainsAndy Chen, Vice President, Match Group. &Research shows that the human brain can only track a set number of relationships…and technology has not helped us increase this limit.&

Chen is referring to the Dunbar number, which says that people can only really maintain a max of some 150 social relationships. Giving users a never-ending list of possible matches on Tinder, then, isn&t helping people feel like they have options & itoverloading the brain.

While turning matchmaking into a game feels a bit dehumanizing & maybe even more so than on Tinder, with its Hot-or-Not-inspired vibe & the team says Crown actually increases the odds, on average, of someone being selected, compared with traditional dating apps.

&When choosing one person over another, there is always a winner. The experience actually encourages a user playing the game to find reasons to say yes,& says Chen.

Crown has been live in a limited beta for a few months, but is now officially launched in L.A. (how appropriate) with more cities to come. For now, users outside L.A. will be matched with those closet to them.

There are today several thousand users on the app, and itorganically growing, Chen says.

Plus, Crown is seeing day-over-day retention rates which are &already as strong& as Match Groupother apps, we&re told.

Sigh.

The app is a free download on iOSonly for now. An Android version is coming, the website says.

As humans, we&ve gotten pretty good at shaping the world around us. We can choose the molecular design of our fruits and vegetables, travel faster and farther and stave off life-threatening diseases with personalized medical care. However, what continues to elude our molding grasp is the airy notion of &time& — how to see further than our present moment, and ultimately how to make the most of it. As it turns out, robots might be the ones that can answer this question.

Computer scientists from the University of Bonn in Germany wrote this week that they were able to design a software that could predict a sequence of events up to five minutes in the future with accuracy between 15 and 40 percent. These values might not seem like much on paper, but researcher Dr. Juergen Gall says it represents a step toward a new area of machine learning that goes beyond single-step prediction.

Although Gallgoal of teaching a system how to understand a sequence of events is not new (after all, this is a primary focus of the fields of machine learning and computer vision), it is unique in its approach. Thus far, research in these fields has focused on the interpretation of a current action or the prediction of an anticipated next action. This was seen recently in the news when apaper from Stanford AI researchers reported designing an algorithm that could achieve up to 90 percent accuracy in its predictions regarding end-of-life care.

When researchers provided the algorithm with data from more than two million palliative-care patient records, it was able to analyze patterns in the data and predict when the patient would pass with high levels of accuracy. However, unlike Gallresearch, this algorithm focused on a retrospective, single prediction.

Accuracy itself is a contested question in the field of machine learning. While it appears impressive on paper to report accuracies ranging upwards of 90 percent, there is debate about the over-inflation of these values through cherry-picking &successful& data in a process called p-hacking.

In their experiment, Gall and his team used hours of video data demonstrating different cooking actions (e.g. frying an egg or tossing a salad) and presented the software with only portions of the action and tasked it with predicting the remaining sequence based on what it had &learned.& Through their approach, Gall hopes the field can take a step closer to true human-machine symbiosis.

&[In the industry] people talk about human robot collaboration but in the end therestill a separation; they&re not really working close together,& says Gall.

Instead of only reacting or anticipating, Gall proposes that, with a proper hardware body, this software could help human workers in industrial settings by intuitively knowing the task and helping them complete it. Even more, Gall sees a purpose for this technology in a domestic setting, as well.

&There are many older people and thereefforts to have this kind of robot for care at home,& says Gall. &In ten years I&m very convinced that service robots [will] support care at home for the elderly.&

The number of Americans over the age of 65 today is approximately 46 million, according to a Population Reference Bureau report, and is predicted to double by the year 2060. Of that population, roughly 1.4 million live in nursing homes according to a 2014 CDC report. The impact that an intuitive software like Gallcould have has been explored in Japan, where just over one-fourth of the countrypopulation is elderly. From PARO, a soft, robotic therapy seal, to the sleek companion robot Pepper from SoftBank Robotics, Japan is beginning to embrace the calm, nurturing assistance of these machines.

With this advance in technology for the elderly also comes the bitter taste that perhaps these technologies will only create further divide between the generations — outsourcing love and care to a machine. For a yet mature industry ithard to say where this path with conclude, but ultimately that is in the hands of developers to decide, not the software or robots they develop. These machines may be getting better at predicting the future, but even to them their fates are still being coded.