Identify Members Who Will Become High Cost Claimants

Business Problem

Healthcare payers often aren’t responsible for paying members’ claims until they reach a certain threshold. Knowing ahead of time which members are most likely to go over this threshold will provide healthcare payers with the ability to intervene and reduce costs.

Intelligent Solution

With AI, you can predict how likely health care members will be to go over a certain cost threshold in the next 12-month period. The model takes the guesswork out of deciding which members to move into programs or to spend marketing dollars on; instead they provide a ranking 0-1 which indicates the members to prioritize.

AI models have the flexibility to be implemented in a number of ways. Because new data may be scored infrequently, non-technical users can use the batch scoring interface or can integrate models into their own applications via the modeling API or the scoring code.

Value Estimation

How would I measure ROI for my use case? 

To measure the effectiveness of the High Cost Claimant model in DataRobot, you can upload an external dataset and use the profit curve and payoff matrices to assign values to the False Positives (cost of intervening with someone who will not go above threshold) and the gain of True Positives (how much is saved, on average, through intervention). This can also be done outside of DataRobot via the API. 

You can also combine the binary classification model that identifies who will exceed the  threshold, with a regression model that predicts the cost of each patient. This is called frequency-severity modeling and is done commonly in insurance use cases.

Problem Framing 

The target variable for this use case is whether or not a member will surpass the defined cost threshold during the given term. This choice in target makes this a binary classification problem. 

The features below represent some of the factors that are important in predicting high cost claims. Beyond these features, we suggest incorporating any additional data your organization may collect that could be relevant to the use case. As you will see later, DataRobot is able to quickly help you differentiate between important and unimportant features. 

Generally, age is most important variable on the Feature Impact graph, along with at least one of the trending variables, followed by one of the regional indicators of life expectancy. Age and trending variables alone should yield a usable model.

Sample Feature List

Data Preparation 

Projects are often built at the level of group or employer. Oftentimes the data contains millions of records with imbalanced positive class (<1%). It may be necessary to downsample the data to use it with DataRobot; when doing this make sure to use stratified sampling on the target variable.

The unit of analysis here is the member at a given point in time. You should be setting up the training data  so that the features are being generated for the months prior to the renewal, and the target will be whether or not each member went over the defined (n) threshold during the given term. 

The trending variables are generated by averaging the most recent n months and calculating the difference for the prior n months. Use 3 and 6 as the different n’s to yield good results.

The variable “TOP 10 DIAGNOSES in LAST 12 MONTHS” is one that can be built by concatenating those fields into one text field. You will have to include a dummy string at the beginning of the field like “force to text” so that DataRobot knows to read it as a text field. There is no need to widen the diagnoses codes and create sparse columns counting the number of times each diagnosis appears. This concatenation trick can be done to save time, save space, and even to slightly improve model accuracy.

Model Training

DataRobot Automated Machine Learning automates many parts of the modeling pipeline. Instead of hand-coding and manually testing dozens of models to find the one that best fits your needs, DataRobot automatically runs dozens of models and finds the most accurate one for you, all in a matter of minutes. In addition to training the models, DataRobot automates other steps in the modeling process such as processing and partitioning the dataset.

While we will jump straight to model deployment, take a look here to see how to use DataRobot from start to finish and how to understand the data science methodologies embedded in its automation.

Here are some notes on modeling for this use case:

• LogLoss should be used to optimize and evaluate the performance of the model. You will see very high accuracy numbers as the data is very imbalanced (e.g., majority class will often be greater than 99%). AUC will be high, usually greater than .90

Decision Environment 

After you are able to find the right model that best learns patterns in your data to predict which members will be high cost, DataRobot makes it easy to deploy the model into your desired decision environment. Decision environments are the ways in which the predictions generated by the model will be consumed by the appropriate stakeholders in your organization, and how these stakeholders will make decisions using the predictions to impact the overall process. 

Decision Maturity 

Automation | Augmentation | Blend

In this use case, we are utilizing DataRobots Automated Machine Learning (AutoML) to identify at-risk claimants. This solution can be fully automated, or augmented with a human in loop if desired. 

Model Deployment

There are a number of ways to implement this model. Because of the low latency of the predictions, performing a drag-and-drop prediction may work for non-technical users. However, this model is often deployed by creating a RESTful API endpoint; in this case, the model is accessed from a BI workstream and the results materialized in a dashboard or application.

Decision Stakeholders

Decision Executors

Population Health Professionals, Underwriters

Decision Managers

Population Health Underwriters

Decision Authors

Data Scientists and Engineers

Decision Process 

Population health experts can identify at-risk patients and reach out to members to make them aware of different services and programs available to them to get services earlier and reduce costs. Additionally, this information can be used by underwriters to price renewals appropriately.

Model Monitoring 

Models should be tracked by evaluating drift and population stability index (PSI). If the model was trained on the same group that the scoring data comes from, then drift is often stable. Problems can arise when the training data is derived from a broader swath of the population that does not include the group being scored. It is also important to keep track of which at-risk patients intervention was performed on as this data will be useful for future modeling. For example, if this program runs for a year then there will be a year’s worth of data available for predicting the likelihood of an intervention resulting in a reduction in costs; this data can be used to test and refine different intervention strategies.  

Implementation Risks

If, for some reason, the scoring group’s data cannot be used to train the model then a good strategy is to perform a stratified sample of the broader group so that the distributions of the predictor variables closely match the scoring data.

From a deployment perspective, the risks are relatively low as the API deployment allows for flexibility to integrate with a wide range of BI environments and applications. Additionally, if manual batch scoring has to be used this is often a viable option that can provide value while an automated solution can be developed.