Technology

Cosmos DB is undoubtedly one of the most interesting products in MicrosoftAzure portfolio. Ita fully managed, globally distributed multi-model database that offers throughput guarantees, a number of different consistency models and high read and write availability guarantees. Now thata mouthful, but basically, it means that developers can build a truly global product, write database updates to Cosmos DB and rest assured that every other user across the world will see those updates within 20 milliseconds or so. And to write their applications, they can pretend that Cosmos DB is a SQL- or MongoDB-compatible database, for example.

CosmosDB officially launched in May 2017, though in many ways itan evolution of Microsoft existing Document DB product, which was far less flexible. Today, a lot of Microsoftown products run on CosmosDB, including the Azure Portal itself, as well as Skype, Office 365 and Xbox.

Today, Microsoft is extending Cosmos DB with the launch of its multi-master replication feature into general availability, as well as support for the Cassandra API, giving developers yet another option to bring existing products to CosmosDB, which in this case are those written for Cassandra.

Microsoft now also promises 99.999 percent read and write availability. Previously, itread availability promise was 99.99 percent. And while that may not seem like a big difference, it does show that after more of a year of operating Cosmos DB with customers, Microsoft now feels more confident that ita highly stable system. In addition, Microsoft is also updating its write latency SLA and now promises less than 10 milliseconds at the 99th percentile.

&If you have write-heavy workloads, spanning multiple geos, and you need this near real-time ingest of your data, this becomes extremely attractive for IoT, web, mobile gaming scenarios,& Microsoft CosmosDB architect and product manager Rimma Nehme told me. She also stressed that she believes MicrosoftSLA definitions are far more stringent than those of its competitors.

The highlight of the update, though, is multi-master replication. &We believe that we&re really the first operational database out there in the marketplace that runs on such a scale and will enable globally scalable multi-master available to the customers,& Nehme said. &The underlying protocols were designed to be multi-master from the very beginning.&

Why is this such a big deal With this, developers can designate every region they run Cosmos DB in as a master in its own right, making for a far more scalable system in terms of being able to write updates to the database. Thereno need to first write to a single master node, which may be far away, and then have that node push the update to every other region. Instead, applications can write to the nearest region, and Cosmos DB handles everything from there. If there are conflicts,the user can decide how those should be resolved based on their own needs.

Nehme noted that all of this still plays well with CosmosDBexisting set of consistency models. If you don&t spend your days thinking about database consistency models, then this may sound arcane, but therea whole area of computer science that focuses on little else but how to best handle a scenario where two users virtually simultaneously try to change the same cell in a distributed database.

Unlike other databases, Cosmos DB allows for a variety of consistency models, ranging from strong to eventual, with three intermediary models. And it actually turns out that most CosmosDB users opt for one of those intermediary models.

Interestingly, when I talked to Leslie Lamport, the Turing award winner who developed some of the fundamental concepts behind these consistency models (and the popular LaTeX document preparation system), he wasn&t all that sure that the developers are making the right choice. &I don&t know whether they really understand the consequences or whether their customers are going to be in for some surprises,& he told me. &If they&re smart, they are getting just the amount of consistency that they need. If they&re not smart, it means they&re trying to gain some efficiency and their users might not be happy about that.& He noted that when you give up strong consistency, itoften hard to understand what exactly is happening.

But strong consistency comes with its drawbacks, too, which leads to higher latency. &For strong consistency there are a certain number of roundtrip message delays that you can&t avoid,& Lamport noted.

The CosmosDB team isn&t just building on some of the fundamental work Lamport did around databases, but italso making extensive use of TLA+, the formal specification language Lamport developed in the late 90s. Microsoft, as well as Amazon and others, are now training their engineers to use TLA+ to describe their algorithms mathematically before they implement them in whatever language they prefer.

&Because [CosmosDB is] a massively complicated system, there is no way to ensure the correctness of it because we are humans, and trying to hold all of these failure conditions and the complexity in any one person— one engineer— head, is impossible,& Microsoft Technical Follow Dharma Shukla noted. &TLA+ is huge in terms of getting the design done correctly, specified and validated using the TLA+ tools even before a single line of code is written. You cover all of those hundreds of thousands of edge cases that can potentially lead to data loss or availability loss, or race conditions that you had never thought about, but that two or three years ago after you have deployed the code can lead to some data corruption for customers. That would be disastrous.&

&Programming languages have a very precise goal, which is to be able to write code. And the thing that I&ve been saying over and over again is that programming is more than just coding,& Lamport added. &Itnot just coding, thatthe easy part of programming. The hard part of programming is getting the algorithms right.&

Lamport also noted that he deliberately chose to make TLA+ look like mathematics, not like another programming languages. &It really forces people to think above the code level,& Lamport noted and added that engineers often tell him that it changes the way they think.

As for those companies that don&t use TLA+ or a similar methodology, Lamport says heworried. &I&m really comforted that [Microsoft] is using TLA+ because I don&t see how anyone could do it without using that kind of mathematical thinking — and I worry about what the other systems that we wind up using built by other organizations — I worry about how reliable they are.&

AWS has its Snowball (and Snowmobile truck), Google Cloud has its data transfer appliance and Microsoft has its Azure Data Box. All of these are physical appliances that allow enterprises to ship lots of data to the cloud by uploading it into these machines and then shipping them to the cloud. MicrosoftAzure Data Box launched intopreview about a year ago and today, the company is announcing a number of updates and adding a few new boxes, too.

First of all, the standard 50-pound, 100-terabyte Data Box is now generally available. If you&ve got a lot of data to transfer to the cloud — or maybe collect a lot of offline data — then FedEx will happily pick this one up and Microsoft will upload the data to Azure and charge you for your storage allotment.

If you&ve got a lot more data, though, then Microsoft now also offers the Azure Data Box Heavy. This new box, which is now in preview, can hold up to one petabyte of data. Microsoft did not say how heavy the Data Box Heavy is, though.

Also new is the Azure Data Box Edge, which is now also in preview. In many ways, this is the most interesting of the additions since it goes well beyond transporting data. As the name implies, Data Box Edge is meant for edge deployments where a company collects data. What makes this version stand out is that itbasically a small data center rack that lets you process data as it comes in. It even includes an FPGA to run AI algorithms at the edge.

Using this box, enterprises can collect the data, transform and analyze it on the box, and then send it to Azure over the network (and not in a truck). Using this, users can cut back on bandwidth cost and don&t have to send all of their data to the cloud for processing.

Also part of the same Data Box family is the Data Box Gateway. This is a virtual appliance, however, that runs on Hyper-V and VMWare and lets users create a data transfer gateway for importing data in Azure. Thatnot quite as interesting as a hardware appliance but useful nonetheless.

Microsoft used its Ignite conference in Orlando, Florida, today to announce a number of new features that are coming to Office 365. Given the companycurrent focus on AI, itno surprise that most of these new featuresare powered by AI in one form or another. That means all of your Office apps, on- and offline, will soon become a little bit easier to use and offer you more assistance.

The most interesting of these new features is called Ideas. With Ideas, Microsoft is rolling out a unified experience in tools like Excel and PowerPoint (with others to follow), that aim to help you save time on everyday tasks. That may be figuring out a specific design and layout in PowerPoint, or finding images for your presentation. In Excel, Ideas may suggest which charts to use or help you discover outliers in your data. This feature is now generally available in Excel and coming to PowerPoint Online as a preview in the near future. Chances are we&ll see it pop up in other Office apps soon. And yes, feel free to insert your own favorite Clippy joke here.

Excel is getting a few additional new AI features, too. Microsoft previously announced data types in Excel at Ignite 2017 — and launched into preview earlier this year. Now this feature, which helps you turn references to stocks and geographies into rich data that you can easily extend and manipulate in a spreadsheet, is generally available. The company is also bringing image recognition to Excel. This nifty new tool lets you convert a picture of a data table into… you guessed it… an Excel file. The name of this tool Data from Picture. Itnow available in public preview.

Microsoft also notes that it is making Excel faster by speeding up lookup-type functions.

Thereanother new feature here, too, that isn&t directly about AI but will still make using Office easier. Microsoft Search is a new search capability that will soon roll out to Bing and Office.com (and later to Microsoft Edge, Windows and Office) and use data from the Microsoft Graph and AI tech from Bing to offer a better and more cohesive search experience.

Microsoft notes that these new search features will also be able to index internal documents so that you can now ask questions like &Can I bring my wife and kids on a work trip& and see an HR document that tells you that you can&t.