How to build an effective drug recommendation system?

Tags
11 December 2020
Complete Guide for CTO & IT Directors
Microservices under X-Ray Three books image Download free ebook

If we want to create a recommendation system for a service or platform we can approach the goal in many ways. Looking at the current trends, in particular the fashion for deep learning, we may be impressed that the use of only very complex algorithms will result in success. However, it is not so. We can apply recommender systems in scenarios where many users interact with many items and the system will help to suggest items that have been hosted by users that are similar.

The challenge with more complex approaches is that they can be sometimes difficult to tune and interpret. In other words, they can be very powerful but require a lot of knowledge to implement properly. Associations analysis from another side is relatively light on the math concepts and easy to explain to laypeople. In addition, it is an unsupervised learning tool that looks for hidden patterns so there is a limited need for data prep and feature engineering. It is a good start for certain cases of data exploration and can facilitate more insightful approaches to data.

A useful technique that we will show here as the first one is called association analysis, which attempts to find common patterns of items in large data sets. The important assumption is that we built an unsupervised rule based on GPI with 10 dig level and Drug Name level. We didn’t want to separate recommendations based on dose; for example, GPI for Lipitor Oral Tablet 10 MG.

The main idea in apriori rules is that we are looking for association rules between drugs, based on the basket (set of products that people buy). It is necessary to remember we are looking for users that buy more than one item. In the case of prescription users (SingleCare clients in this case), the number of drugs that they bought is not so big as in everyday situations in the supermarket. Therefore, we first chose only the customers that bought more than one drug. For simplicity, we created an artificial basket based on one year of user history. Based on the experiment and the results, we are choosing the customers who buy more than 4 drugs per year. Keep in mind that after preprocessing, we are creating rules from 3 important components: member_id, drug_name, count_of_drug (for each drug & client separately).

Important terminology:

Support is the relative frequency that the rules show up. In many instances, you may want to look for high support in order to make sure it is a useful relationship. However, there may be instances where low support is useful if you are trying to find “hidden” relationships, one of the measures of interest. These relationships describe the usefulness and certainty of the rules. 5% Support means 5% of transactions in the database follow the rule.

Confidence is a measure of the reliability of the rule. Confidence of 0.5 in the above example would mean that in 50% of the cases where amoxicillin and vitamin C were purchased, the purchase also included ibuprofen. For product recommendation, a 50% confidence may be perfectly acceptable but in a medical situation, this level may not be high enough. Confidence: A confidence of 60% means that 60% of the customers who purchased amoxicillin and vitamin C also bought ibuprofen.

Lift is the ratio of the observed support to that expected if the two rules were independent. The basic rule of thumb is that a lift value close to 1 means the rules were completely independent. Lift values > 1 are generally more “interesting” and could be indicative of a useful rule pattern.

The most interesting part of the algorithm is that rules have the direction where one drug is antecedents and the other is consequents. A patient who buys a cancer drug (antecedents) also buys a painkiller (consequents) indicates a strong rule. Inversely, it doesn’t work: not everyone suffering from pain has cancer. We are able to filter this kind of recommendation automatically by apriori algorithm parameters. Now based on the strongest rules (filtered by lift metrics), we are able to achieve some interesting results:

  1. more than 60% of the rules are related to popular drugs (first 50 drugs for all drugs available).
  2. more than 50% of the drugs that are recommended are cheaper for clients.  
  3. over 40% of strong apriori rules recommended by apriori increase profitability for a company.

Michał Olędzki

Latest Posts
bots with python

Bots with Python 101

As we continue to embrace the digital age, we encounter countless innovative solutions that improve our daily lives, making mundane tasks more efficient, or even automating them entirely. One such innovative solution is the ‘bot’, a broad term that has various definitions depending on the context in which it is used. In its essence, a […]

/
product roadmap example

Which Way To Go – Product Roadmap Example And Insights

A quick overview of product roadmaps, not only for product managers. Briefly, What Is A Product Roadmap? A product roadmap is a visual representation that outlines a product development and evolution over a defined period, serving as a communication tool to align stakeholders around the product’s direction, goals, and milestones. In general terms, the roadmap […]

/
how to secure bluetooth devices

How to Secure Bluetooth Devices?

In today’s interconnected digital era, our lives are continuously shaped, molded, and improved by the innovative technologies we embrace. Every once in a while, a technological advancement emerges that becomes so deeply embedded in our routines that it’s hard to imagine a world without it. Bluetooth, a wireless communication protocol, is certainly one such marvel. […]

/
ux review

UX review: How to Perform a Usability Audit and Why it’s Important

User experience is crucial for the success of any digital product. A properly conducted UX review can help you identify flaws in your app’s or website’s design and fix them. Find out how a usability audit works and what you can gain from it. It’s hard to overestimate the impact UX has on business. According […]

/
prototype vs proof of concept

Prototype vs Proof of Concept: A Dive into Digital Product Development

In the realm of product development and innovation, two terms are often bandied about with great fervor – ‘Prototype’ and ‘Proof of Concept’ (POC). These methodologies play pivotal roles in transforming an abstract idea into a tangible product, serving as critical stages in the pathway to commercialization. However, these terms are frequently misunderstood, often interchanged […]

/
What is Python Used for in Finance

What is Python Used for in Finance?

As we delve into the digital age, the fields of finance and technology have become intricately intertwined, birthing an innovative hybrid sector known as financial technology, or “Fintech.” As this sector expands and evolves, one programming language stands at its epicenter, powering the development and execution of numerous innovative applications — Python. In a world […]

/
Related posts
python vs scala

Scala vs Python: What’s Better for Big Data?

In the world of big data processing and analytics, choosing the right programming language is crucial for efficient data management and effective decision-making. Python and Scala are two popular programming languages used in data science projects and processing, each with its unique strengths and features. This article will explore the key differences between Python and […]

/
predictive analytics retail

Predictive Analytics: Retail Industry Use Cases

Retail businesses have always been driven by data, whether it’s sales figures, inventory levels, or customer behavior. With the rise of predictive analytics, retailers have the ability to analyze large volumes of data to uncover data-driven insights and patterns that can help them make more informed decisions. Predictive analytics is a branch of advanced analytics […]

/

Python Web Application Examples. Top 7 Cases

Python lies at the heart of many leading web applications. Businesses and programmers love this language for its simplicity which, paradoxically, facilitates the development of very complex systems. Find out how top big tech companies use Python in their platforms. Python is the language of choice for data scientists, machine learning experts, and backend developers. […]

/

Economy business cycle prediction using LSTM neural networks

Economy as a whole goes through business cycles that consist of alternating expansions and contractions. It is not immediately obvious  what cycle the economy is currently in; it is difficult to tell if it is at the peak of an expansion or the trough of a contraction. The ability to predict economy cycles would be […]

/

How to build an effective drug recommendation system?

If we want to create a recommendation system for a service or platform we can approach the goal in many ways. Looking at the current trends, in particular the fashion for deep learning, we may be impressed that the use of only very complex algorithms will result in success. However, it is not so. We can apply […]

/
Talk with experts

We look forward to hearing from you to start expanding your business together.

Email icon [email protected] Phone icon +1 (888) 413 3806