AI Platforms and Tools, Big Data Analytics

The Distribution of Everything

By Adam Mayer, Qlik Technical Product Marketing

On the winds of change is the trend towards cloud-native container based microservices architecture which ultimately means that organisations embracing this will be able to distribute workloads, processes and indeed data throughout the enterprise and even across clouds, as and when they need to, with much more flexibility than before.

According to 451 Research – 45% of organisations today are either actively using container technologies or are in the discovery stage, with 7% planning to implement in the next 12 months.1

This is only set to grow, in fact IDC predict that by 2022, 90% of all new apps will feature microservices architectures that improve the ability to design, debug, update, and leverage third-party code.

But what’s all this have to do with analytics? I hear you cry, well a few years ago we started talking about hybrid cloud and partners like Qlik have been executing and delivering on their own exciting multi-cloud strategy with containerized microservices architecture at the heart of it. In this blog I want to start demystifying what this all means.

Let’s start with cloud first. Just because something is cloud-native does not mean it was born in and must only live in the cloud. In Qlik’s case they decided to lead with cloud, rather than retrofitting the product, they have built a cloud-native architecture from the ground up. A modern web-based platform can handle the demands of more elasticity and scale; portability across different environments for multi-cloud; and allow for easy changes/shifts in deployment. You can deploy Qlik on-premise, in a public cloud, in your private cloud or virtual private cloud, on Qlik’s own cloud with our Qlik Cloud Services and anywhere in-between.

While it’s true that more organisations are moving more towards cloud, that doesn’t mean you have to move everything on the cloud, including all your data. In this post GDPR world there will always be cases where your more sensitive data must stay on-premise, behind the fire wall, and absolutely, positively cannot leave the room. That’s okay too but you don’t want to be forced to analyse it in isolation.

For most organisations these days it’s no longer just a single cloud vendor they partner with either, there can be several cloud vendors in the mix – hence the need for a multi-cloud strategy and why Qlik has a strong ecosystem of technology partners including the same cloud vendors you use.

Now let’s look at why many organizations see containerized microservices architecture as the way forward. There are many benefits, here are just a few;

  • Software applications can be split out into smaller isolated services with less dependencies on operating systems and servers making them smaller in size (MBs) and faster to spin up and easier to manage change, compared to virtual machines.
  • Optimized elastic infrastructure offering high availability that can be scaled-out rapidly
  • Automated capabilities to manage and orchestrate large and complex applications with ease
  • Portability to seamlessly move between environments (Dev>Test>Production), on-premise, cloud and even between clouds, helping to avoid vendor lock in
  • Security benefits as it can add another layer of isolation between your application and the host environment, without requiring you to run a full virtual server.

Technologies such as Docker and Kubernetes are becoming synonymous with containerized microservices. Docker allows you to build, ship and run applications in containers. Kubernetes automates the management of containers and is great for scenarios where there are many containers and complex models.

1Source: 451Research: Voice of the Enterprise: Digital Pulse, Budgets and Outlooks 2018

Would you like to join the Data Architects und Data Engineering DACH community?

Join ScaleUp 360° Smart Data Architectures & Data Engineering