Technology

Cloudflare refuses to regulate the sadistic content it optimizes and protects online.

A group of computer vision researchers from ETH Zurich want to do their bit to enhance AI development on smartphones. To wit: They&ve created a benchmark system for assessing the performance of several major neural network architectures used for common AI tasks.

They&re hoping it will be useful to other AI researchers but also to chipmakers (by helping them get competitive insights); Android developers (to see how fast their AI models will run on different devices); and, well, to phone nerds — such as by showing whether or not a particular device contains the necessary drivers for AI accelerators. (And, therefore, whether or not they should believe a companymarketing messages.)

The app, called AI Benchmark, is available for download on Google Play and can run on any devicewith Android 4.1 or higher — generating a score the researchers describe as a &final verdict& of the deviceAI performance.

AI tasks being assessed by their benchmark system include image classification, face recognition, image deblurring, image super-resolution, photo enhancement or segmentation.

They are even testingsome algorithms used in autonomous driving systems, though therenot really any practical purpose for doing that at this point. Not yet anyway. (Looking down the road, the researchers say itnot clear what hardware platform will be used for autonomous driving — and they suggest it&quite possible& mobile processors will, in future, become fast enough to be used for this task. So they&re at least prepped for that possibility.)

The app also includes visualizations of the algorithms& output to help users assess the results and get a feel for the current state-of-the-art in various AI fields.

The researchers hope their score will become a universally accepted metric — similar to DxOMark that is used for evaluating camera performance — and all algorithms included in the benchmark are open source. The current ranking of different smartphones and mobile processors is available on the projectwebpage.

The benchmark system and app was around three months in development, says AI researcher and developer Andrey Ignatov.

He explains that the score being displayed reflects two main aspects: The SoCspeed and available RAM.

&Letconsider two devices: one with a score of 6000 and one with a score of 200. If some AI algorithm will run on the first device for 5 seconds, then this means that on the second device this will take about 30 times longer, i.e. almost 2.5 minutes. And if we are thinking about applications like face recognition this is not just about the speed, but about the applicability of the approach: Nobody will wait 10 seconds till their phone will be trying to recognize them.

&The same is about memory: The larger is the network/input image — the more RAM is needed to process it. If the phone has small amount of RAM that is e.g. only enough to enhance 0.3MP photo, then this enhancement will be clearly useless, but if it can do the same job for Full HD images — this opens up much wider possibilities. So, basically the higher score — the more complex algorithms can be used / larger images can be processed / it will take less time to do this.&

Discussing the idea for the benchmark, Ignatov says the lab is &tightly bound& to both research and industry — so&at some point we became curious about what are the limitations of running the recent AI algorithms on smartphones&.

&Since there was no information about this (currently, all AI algorithms are running remotely on the servers, not on your device, except for some built-in apps integrated in phonefirmware), we decided to develop our own tool that will clearly show the performance and capabilities of each device,& he adds.

&We can say that we are quite satisfied with the obtained results — despite all current problems, the industry is clearly moving towards using AI on smartphones, and we also hope that our efforts will help to accelerate this movement and give some useful information for other members participating in this development.&

After building the benchmarking system and collating scores on a bunch of Android devices, Ignatov sums up the current situation of AI on smartphones as &both interesting and absurd&.

For example, the team found that devices running Qualcomm chips weren&t the clear winners they&d imagined — i.e. based on the companypromotional materials about Snapdragon845 AI capabilities and 8x performance acceleration.

&It turned out that this acceleration is available only for ‘quantized& networks that currently cannot be deployed on the phones, thus for ‘normal& networks you won&t get any acceleration at all,& he says. &The saddest thing is that actually they can theoretically provide acceleration for the latter networks too, but they just haven&t implemented the appropriated drivers yet, and the only possible way to get this acceleration now is to use Snapdragonproprietary SDK available for their own processors only. As a result — if you are developing an app that is using AI, you won&t get any acceleration on SnapdragonSoCs, unless you are developing it for their processors only.&

Whereas the researchers found that HuaweiKirin970 CPU — which is technically even slower than Snapdragon 636 — offered a surprisingly strong performance.

&Their integrated NPU gives almost 10x acceleration for Neural Networks, and thus even the most powerful phone CPUs and GPUs can&t compete with it,& says Ignatov. &Additionally, Huawei P20/P20 Pro are the only smartphones on the market running Android 8.1 that are currently providing AI acceleration, all other phones will get this support only in Android 9 or later.&

Itnot all great news for Huawei phone owners, though, as Ignatov says the NPUdoesn&t provide acceleration for ‘quantized& networks (though he notes the company has promised to add this support by the end of this year); and also it uses its own RAM — which is &quite limited& in size, and therefore you &can&t process large images with it&…

&We would say that if they solve these two issues — most likely nobody will be able to compete with them within the following year(s),& he suggests, though he also emphasizes that this assessmentonly refers to the one SoC, noting that Huaweiprocessors don&t have the NPU module.

For Samsung processors, the researchers flag up thatall the companydevices are still running Android 8.0 but AI acceleration is only available starting from Android 8.1 and above. Natch.

They also found CPU performance could &vary quite significantly& — up to 50% on the same Samsung device — because of throttling and power optimization logic. Which would then have a knock on impact on AI performance.

ForMediatek, the researchers found the chipmaker isproviding acceleration for both ‘quantized& and ‘normal& networks — which means it can reach the performance of &top CPUs&.

But, on the flip side,Ignatov calls out the companyslogan — that it&Leading the Edge-AI Technology Revolution& — dubbing it ¬hing more than their dream&, and adding: &Even the aforementioned Samsunglatest Exynos CPU can slightly outperform it without using any acceleration at all, not to mention Huawei with its Kirin970 NPU.&

&In summary: Snapdragon — can theoretically provide good results, but are lacking the drivers; Huawei — quite outstanding results now and most probably in the nearest future; Samsung — no acceleration support now (most likely this will change soon since they are now developing their own AI Chip), but powerful CPUs; Mediatek — good results for mid-range devices, but definitely no breakthrough.&

Italso worth noting that some of the results were obtained onprototype samples, rather than shipped smartphones, so haven&t yet been included in the benchmark table on the teamwebsite.

&We will wait till the devices with final firmware will come to the market since some changes might still be introduced,& he adds.

For more on the pros and cons of AI-powered smartphone features check out our article from earlier this year.

Market research firm GlobalWebIndex, which provides consumer insight data for marketing purposes for customers including Google, Spotify, WPP, IPG and Omnicom Group, has closed a $40 million Series A round. The funding is its first VC raise, almost a decade after the business was founded.

The investment comesfromNew York-based growth fundStripes Group, along with a number of other unnamed data, software and consumer technology companies. GWI says it will be used to accelerate product development and for international expansion, including in the U.S. and Asia.

The company isbased in London but has recently opened offices in New York City and Los Angeles, as well as having technology hubs in locations across Europe.

With the new funding it says itplanning to open more international offices across the Americas and Asia Pacific tosupport a client base which spans more than 80 countries.

But why take VC now &After nine years with no funding and seeing phenomenal growth and expansion, we are still seeing an increasing demand for our data, especially from companies that we haven&t traditionally sold to,& says CEO Tom Smith.

&The new funding will support our product development and hiring efforts so we can establish ourselves as the go to platform for digital consumer insights for the marketing industry.&

The company believes itpositioned itself on the right side of digital history, having chosen an opt-in, survey-based route for gathering a chunk of its consumer data for market research purposes — putting detailed questions to its global panel of 22 million web users from whom itgaining up front consent to their data being processed.

Europenew data protection framework, GDPR, is explicit on the need for consent to be informed, specific and freely given if thatyour legal basis for processing peoplepersonal data.

On the product development front, GWI says itworkingto develop new ways of collecting consumer data — having developed a proprietary,device-agnostic &messenger-style survey tool& which Smith says allows respondents to &answer questions at times and in formats which suit them&.

&Itabout putting the consumer first — not just in how their data is used, but how you run the survey itself. With this new approach we hope to be able to return survey results faster to our clients, so they can make quick business decisions based on insights retrieved from our tried and tested methodology,& he adds.

&Years before GDPR, we wanted to be respectful to those who take one of our surveys, meaning that they see consumer-centric privacy and consent notices which use easy-to-understand language to outline what we do, why itimportant to our clients, and what their responses will be used for.&

&We ask our respondents a wide range of questions relating to their digital lives and lifestyles,& he continues. &This covers everything from their social media, device, media consumption and online behaviors to their interactions with brands, their attitudes and their daily life. For us, itkey to understand not just what people are doing online, but the attitudes, motivations and beliefs which drive this.&

GWIprimary product offering is its core study — which is fielded in 44 countries, and which it says contains 25,000 data points on 22M+ &connected consumers&.

&Each year we interview hundreds of thousands of representative individuals about their digital lives. The aggregated results of this are made available on our industry-leading platform on a syndicated basis, where clients can build and then analyze any audience they like. For example, you could look at Older vs Younger Millennials, Mums vs Dads, Instagrammers vs Snapchatters — in each case, understanding which behaviors and attitudes are unique to each group,& explains Smith.

It also offers a range of custom services to supplement that core survey-based market intelligence data.

&Many clients use this offering to ask follow-up questions to our respondents, allowing them to overlay the answers to their bespoke questions with the 25,000 data points contained in the core data set,& he says, adding: &We also offer a wide range of other research solutions such as brand tracking, path to purchase journey, ad-effectiveness, concept testing, website analytics and more. Here the sheer scale of our panel — currently at 22M consumers — is a real differentiator.&

All survey respondents are compensated for their time, according to Smith — in different ways, depending on the market, but including via monetary payments, vouchers, loyalty points, charitable donations etc.

One thing to note is that GWI does also use cookie-based tracking to gather data less directly — including by working with partners. So it is also reliant on third parties obtaining consent to data processing, and must therefore rely on those partners to cleanly and fairly obtain consent for this portion of its market intelligence activity.

&In some instances, the cookies we use as part of our research are dropped by the research panels we work with, rather than by GlobalWebIndex itself. In these instances, all such panels are required to obtain consent in GDPR-compliant manners,& says Smith on this.

The company tells TechCrunch it uses cookie tracking to enrich its core survey data, and only uses cookies to trackthe behaviors of its panelists — and only those who have actively opted in to this type of tracking.

&Our analytics technology connects the data we collect through surveys to the behaviors of our panel on client websites and connected properties, as well as their exposure to digital advertising campaigns. In this way, we can leverage the 25,000+ data points we collect through panelist interviews to their browsing behaviors and online activities, providing a unique connection between brand engagement and the attributes, motivations and interests of our clients& target audiences.&

&The GDPR was welcome validation of the approach we have always adopted, whereby respect for the consumer is central,& adds Smith. &Given that we have a direct relationship with the individuals who complete our surveys, we have an amazing opportunity to outline there and then what we want to do with their responses, and to gain their explicit consent for the use of cookies.

&The nature of survey-based market research also means that the consumers know which types of data they are providing, and can decline to answer certain questions if they so choose.&

So why does a company like Google — which has access to vast, global consumer data stores of its own, gathered from its own products and via a network of online tracking cookies and pixels — need GWImarket intelligence

&Many of our clients have their own proprietary sources of consumer data but turn to GlobalWebIndex because of the robust, global, independent view it offers on consumer behaviors,& Smith responds on this. &Our ability to give a 360-degree view on the consumer is particularly valuable, with our data providing a unique cross-device, cross-platform perspective.&

On the competition front, he names the likes of Nielsen, YouGov, Comscore, Kantar, GfK and Simmons. &We provide a global view which is consistent across countries, unlike some alternatives which operate different surveys at different times and then patch them together,& he argues.

He also touts the &incredibly rich view of the consumerdigital life& the GWI consumer panel is able to generate, given the number of data points itgathering.

&We have this depth on all of our respondents, whereas some other sources will only have small pockets of data on each individual,& he claims. &We track behaviours and attitudes from the consumerperspective. There are no inferences, no modeling, no assumptions based on browsing. This is how the consumer acts and feels from their own point of view.

&We provide up-to-the minute data which tells you what people are doing in the here and now. Our quarterly releases will soon move to monthly, and from there we&ll develop a real-time version of our data set.&

&Our ability to re-interview our respondents is hugely important for clients, as it allows them to sync their bespoke questions with the 25,000 data points from the core study. Ita hugely quick and efficient way to gain a rich understanding of your target audience,& he adds.