Virtually every major organization these days is a data company. Enterprises are increasingly leveraging data and data apps for better applications, services, new insights and to manage their business better. This rings true for all industries, but it’s particularly significant in healthcare. According to IDC, healthcare will see data grow faster than any other vertical through 2025.
Modern data apps are at the core of the cutting-edge developments that are redefining healthcare, such as genomic mapping, breakthrough medical imaging, IoT, advanced diagnostics and every type of next- generation research. To say these apps are mission critical would be an understatement – they’re ultimately saving lives. But as these apps have become more powerful, they’ve also grown more complex and difficult to manage.
At Clearsense, we provide a healthcare data platform that allows clients to import raw data into a private cloud and then leverage that data for a variety of apps and for analytics that yield clinical, operational, financial and research insights. The Clearsense data management platform ingests legacy and contemporary data from various real-time and batch-oriented healthcare systems and devices, transforming and delivering this data for immediate consumption by reporting, analytics and AI platforms.
With life or death potentially at stake, uptime and reliability are critical for data apps used by healthcare organizations.
It’s extremely difficult for these companies to manage massive volumes of streaming data from highly distributed, disparate endpoints. Streaming data is prevalent in healthcare settings; it’s complex, comes in enormous quantities and is constantly flowing. As a result, streaming data apps often fail. Think of all the IoT apps and devices being used in a hospital. For example, a smart vital sign monitoring platform that not only tells staff a patient’s current vitals but also can warn them of an upcoming cardiac arrest based on their heartbeat patterns analyzed against historic data and from other patients. Such an app relies on a constant stream of data and must deliver insights in real time. This is exactly the type of app that’s liable to fail often, sometimes with dire consequences.
Streaming apps have two fundamental pieces, a storage component and a compute component. On the storage side, they receive data from a stream store, such as Kafka and Kudu. On the compute side, systems such as Spark Streaming and Flink absorb data from the stream store and run analysis in real time before sending to dashboards and databases. All of these pieces are individually distributed components that are connected by the application. When one of these pieces fails, it causes the whole app to break or slow down.
The default way that data operations teams address these app failures is through manual assessment. Someone has to comb through huge quantities of raw log data and then do weeks of testing trial and error to determine which part of the app failed and then understand why it failed. This sort of approach takes near Ph.D.-level specialization in technologies like Spark or Kafka to accurately make sense of the raw data. Even then, it’s a slow, expensive process that leaves a critical data app down or slow for weeks while the problem is being diagnosed. Furthermore, manual approaches make it impossible to anticipate app failures, leaving DataOps teams doing nothing but reacting to problems.
In addition to the difficulty of managing modern data apps, healthcare companies also face a challenge in predicting the resources and infrastructure (such as servers) that they’ll need to deploy in order to support these apps in the near future. It’s clear they’re relying more and more on data apps, but it’s difficult to forecast exactly how much. An organization’s demand for big data is very elastic – sometimes usage grows for weeks, but then suddenly plateaus or goes down before surging back up. IT can try to guess how much they should budget for new infrastructure in a year, but if they buy too much, they’ve wasted CAPEX, while if they buy too little, their data apps will bottleneck.
We witnessed firsthand how healthcare organizations struggled to manually manage their complex streaming data apps. Then we adopted a full stack DataOps platform and saw those issues disappear. Unravel Data makes heavy use of AI to solve operational problems: the platform features robust AI capabilities that provide automated fixes and recommendations, taking all the manual guesswork out of the equation. The AIOps capabilities draw on rich historical data to warn when a problem may be coming and recommend fixes in advance. These AI features also inform users where they could be getting better performance out of an app and how they can re-configure it to improve it. In the event that an app does fail or slow down, Unravel diagnoses the problem and automatically corrects it. No digging through raw logs, no weeks of trial and error.
This AIOps capability is the real differentiator, allowing us to be proactive in managing a highly distributed modern data stack. Clearsense had tried an alternative solution before Unravel, but it only provided basic monitoring capabilities rather than true management, making it impossible to anticipate and avoid problems in advance. With its deep AI and machine learning capabilities, Unravel goes beyond simple monitoring and enables healthcare organizations to be proactive in optimizing and operationalizing their data apps.
The market offers a few good platforms for managing apps, such traditional APM tools, but Unravel is unique in that it was the first platform built specifically for the modern big data stack. Data apps have much different needs than common web apps. With that specialized background, Unravel supports a broad ecosystem of big data technologies that healthcare companies use to build their mission critical data apps. One of the biggest challenges of managing a modern data stack is its highly distributed nature, with data pouring in from endpoints spread everywhere. This is especially true in a healthcare setting. Luckily, Unravel provides a simple, comprehensive view of all those endpoints.
The platform also leverages its AI for addressing the CAPEX infrastructure challenge mentioned above. Based on a variety of factors, Unravel gives an accurate assessment of how many compute and storage resources you’ll need to support your data stack, whether in six months or two years, in the public cloud or in your own data center.
Data driven apps are increasingly the norm and the source of business value, and they’re also only going to become more difficult to manage on your own. These apps have a huge impact on any business’s bottom line, but in healthcare, they can mean life or death. Attempting to manually manage data deployments is simply not tenable in this industry. Unravel’s AI driven DataOps platform radically simplified our capability to serve the needs of our client data needs and significantly boosted the value they get from these apps while cutting costs. After using an AI-based platform to automatically fix, optimize and re-configure data apps, it’s hard to imagine that we ever tried to do it manually.
About The Author
Charles is Co-Founder & Chief Innovation Officer of Clearsense, LLC. He is known as a notable expert clinical information systems specialist offering 25-plus years of strategic leadership. Charles started his career as a bedside trauma nurse at LA County, USC Medical Center. After spending much of that time with medical devices and medical information systems, he began to witness many gaps in medical technology, later transitioning to focus on the IT side of healthcare full time.
