Skip to content

added CostofCare files #984

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Ashwin-ap wants to merge 2 commits into
base: main
Choose a base branch
from
Open

added CostofCare files #984

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f66ab24
added CostofCare files
Aug 8, 2025
f89f66a
latest oct changes in coc
Ashwin-ap Oct 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
364 changes: 364 additions & 0 deletions UseCases/CostOfCare_Healthcare/CostOfCare_Healthcare.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,364 @@
{
"cells": [
{
"cell_type": "markdown",
"id": "59134b4c-1ed7-4552-ac6a-5293da3f4367",
"metadata": {},
"source": [
"<header>\n",
" <p style='font-size:36px;font-family:Arial; color:#F0F0F0; background-color: #00233c; padding-left: 20pt; padding-top: 20pt;padding-bottom: 10pt; padding-right: 20pt;'>\n",
" Cost Of Care Analytics Demo\n",
" <br>\n",
" <img id=\"teradata-logo\" src=\"https://storage.googleapis.com/clearscape_analytics_demo_data/DEMO_Logo/teradata.svg\" alt=\"Teradata\" style=\"width: 125px; height: auto; margin-top: 20pt;\">\n",
" </p>\n",
"</header>"
]
},
{
"cell_type": "markdown",
"id": "bad1bb80-35d4-4787-a1ae-76d9c161d9bf",
"metadata": {},
"source": [
"<p style = 'font-size:20px;font-family:Arial'><b>Introduction:</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial'>With rising healthcare expenditures, there's a growing urgency to track, analyze, and optimize cost drivers across emergency room (ER), imaging, surgical, and mental health services. Traditional cost auditing methods are retrospective, siloed, and lack the granularity to enable proactive cost containment. By leveraging advanced analytics and integrated hospital datasets, a scalable, dynamic, and contextual cost intelligence solution can be achieved. However, key challenges remain: \n",
"</p>\n",
"<ul style = 'font-size:16px;font-family:Arial'>\n",
" <li>Fragmented cost data across procedures, diagnoses, and provider systems.</li>\n",
" <li>Lack of context linking ER visits, imaging, surgeries, and mental health costs to patient pathways.</li>\n",
" <li>Limited visual insights into cost trends across time, region, and condition severity.</li>\n",
" <li>Need for scalable, in-database analytics to handle massive healthcare datasets.</li>\n",
" \n",
"</ul>\n",
"<p style = 'font-size:18px;font-family:Arial'><b>Proposed Solution:</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial'>The proposed solution leverages Teradata Vantage as an intelligent analytics platform to consolidate, process, and visualize healthcare cost data across clinical domains. Structured datasets from emergency room (ER) visits, imaging studies, surgical procedures, and mental health conditions were ingested into the Vantage environment </p>\n",
"\n",
"<ul style = 'font-size:16px;font-family:Arial'>\n",
" <li><b>Annual-Across Dashboard:</b><br>\n",
" Tracks the total and proportionate number of ER visits year-over-year, highlighting avoidable and treatable cases. This allows stakeholders to identify cost reduction opportunities by focusing on potentially preventable ER encounters.</li>\n",
" <li><b>Mental Health Dashboard:</b><br>\n",
" Displays the yearly trend of mental health–related diagnoses, illustrating a sharp increase in recent years. This trend emphasizes the growing importance of behavioral health management and targeted interventions to contain rising costs.</li>\n",
" <li><b>Top Diagnoses Dashboard:</b><br>\n",
" Analyzes the top emergency visit drivers and their associated per-1,000-patient rates and Net Paid PMPM (Per Member Per Month) costs. Conditions such as cancer, residual complications, and viral infections emerge as leading cost contributors—helping prioritize care management programs.</li>\n",
" <li><b>X-Ray / Echo Tracker Dashboard:</b><br>\n",
" Provides patient- and provider-level utilization analytics for imaging tests like X-rays and echocardiograms. These insights identify overutilization patterns and support efficiency initiatives in radiology departments.</li>\n",
" <li><b>Imaging Studies Dashboard:</b><br>\n",
" Aggregates annual and monthly imaging volumes and costs. Trends reveal a substantial growth in imaging utilization and related expenditures, driving the need for predictive capacity planning and budget alignment.</li>\n",
"</ul>\n",
"\n",
"<p style = 'font-size:16px;font-family:Arial'>Using these curated datasets, interactive dashboards were developed in Superset to provide a unified “Cost of Care” view that enables value-based decision-making:\n",
"</p>\n",
"<p style = 'font-size:18px;font-family:Arial'><b>Benefits:</b></p>\n",
"<ul style = 'font-size:16px;font-family:Arial'>\n",
" <li>End-to-end cost visibility across all major healthcare services—ER, imaging, surgery, and mental health.</li>\n",
" <li>Identification of high-cost hospitals and procedures with spatial and temporal granularity.</li>\n",
" <li>Scalable feature engineering and analytics via Teradata’s in-database capabilities.</li>\n",
" <li>Supports hospital value-based purchasing (HBVP) and patient-centric care strategies.</li>\n",
" <li>Interactive and user-friendly dashboards enable deep dives into cost outliers and overutilization patterns.</li>\n",
"</ul>\n"
]
},
{
"cell_type": "markdown",
"id": "059e24ad-9108-48f7-88c4-b577153593a9",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;\">\n",
"<p style = 'font-size:20px;font-family:Arial;'><b>1. Connect to Vantage, Import python packages and explore the dataset</b></p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "89b8c63c-94d8-4b6e-b29b-f0208beb8d23",
"metadata": {},
"outputs": [],
"source": [
"#import libraries\n",
"import getpass\n",
"from teradataml import *\n",
"\n",
"import warnings\n",
"warnings.filterwarnings('ignore')\n",
"warnings.simplefilter(action='ignore', category=DeprecationWarning)\n",
"warnings.simplefilter(action='ignore', category=RuntimeWarning)\n",
"warnings.simplefilter(action='ignore', category=FutureWarning)\n",
"\n",
"display.max_rows=5"
]
},
{
"cell_type": "markdown",
"id": "4f91da96-35e7-4cc2-aa5b-3a3eb71b543d",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;\">\n",
"<b style = 'font-size:18px;font-family:Arial;'> 1.1 Connect to Vantage</b>\n",
"<p style = 'font-size:16px;font-family:Arial;'>We will be prompted to provide the password. We will enter the password, press the Enter key, and then use the down arrow to go to the next cell.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cb703946-e2ba-446f-aae5-e2a34d699faa",
"metadata": {},
"outputs": [],
"source": [
"%run -i ../startup.ipynb\n",
"eng = create_context(host = 'host.docker.internal', username='demo_user', password = password)\n",
"print(eng)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ef79de0d-ce15-4d28-a642-5eda67c7c56e",
"metadata": {},
"outputs": [],
"source": [
"%%capture\n",
"execute_sql('''SET query_band='DEMO=FF_CostOfCare_Healthcare.ipynb;' UPDATE FOR SESSION; ''')"
]
},
{
"cell_type": "markdown",
"id": "2aea1cce-2002-4169-8e48-39a04c77eb90",
"metadata": {},
"source": [
"<p style = 'font-size:18px;font-family:Arial'> <b>Getting Data for This Demo</b></p>\n",
"<p style = 'font-size:16px;font-family:Arial'>We have provided data for this demo on cloud storage. We have the option of either running the demo using foreign tables to access the data without using any storage on our environment or downloading the data to local storage, which may yield somewhat faster execution. However, we need to consider available storage. There are two statements in the following cell, and one is commented out. We may switch which mode we choose by changing the comment string.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "79059da9-5a68-4860-8c55-b530d85f67fd",
"metadata": {},
"outputs": [],
"source": [
"%run -i ../run_procedure.py \"call get_data('DEMO_HealthcareCOC_cloud');\"\n",
"# takes about 3 minutes\n",
"#%run -i ../run_procedure.py \"call get_data('DEMO_HealthcareCOC_local');\" \n",
"# takes about 15minutes"
]
},
{
"cell_type": "markdown",
"id": "73b03eb2-acc3-4289-8a1c-98aeca10d2d6",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial'>Optional step – We should execute the below step only if we want to see the status of databases/tables created and space used.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "09848365-4c41-4c9d-8db4-a57249bf6dd8",
"metadata": {},
"outputs": [],
"source": [
"%run -i ../run_procedure.py \"call space_report();\""
]
},
{
"cell_type": "markdown",
"id": "6940e9db-d80c-4cae-841b-98dd565c9038",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;\">\n",
"<p style = 'font-size:20px;font-family:Arial'><b>2. Data Exploration </b></p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "501b0984-378e-41b5-915a-f9beb8e327f2",
"metadata": {},
"outputs": [],
"source": [
"db_list_tables('DEMO_HealthcareCOC')"
]
},
{
"cell_type": "markdown",
"id": "39f95466-1c7f-45d7-8533-ff2028bfc40f",
"metadata": {},
"source": [
"<p style = 'font-size:18px;font-family:Arial'><b>Data Sources and Schema Overview:</b></p>\n",
"<ul style = 'font-size:16px;font-family:Arial'>\n",
" <li><b>Va_Encounters:</b>\n",
" Contains detailed records of patient encounters, including provider, payer, costs, and visit type.</li>\n",
" <li><b>Va_Medications:</b>\n",
" Tracks prescribed medications with cost, payer coverage, and associated encounter information.</li>\n",
" <li><b>Va_Procedures:</b>\n",
" Lists performed medical procedures with codes, descriptions, and base cost data.</li>\n",
" <li><b>Va_Conditions:</b>\n",
" Records diagnosed patient conditions with start/stop dates, codes, and encounter references.</li>\n",
" <li><b>Va_Immunizations:</b>\n",
" Logs administered immunizations with associated patient, encounter, and cost details.</li>\n",
" <li><b>Va_Providers:</b>\n",
" Holds provider demographic, specialty, and utilization information for encounter linkage.</li>\n",
" <li><b>va_payers:</b>\n",
" Stores payer or insurance organization details, coverage amounts, and financial performance metrics.</li>\n",
" <li><b>va_bk_diag_nyu:</b>\n",
" Diagnostic mapping table classifying conditions by avoidability and emergency department category.</li>\n",
" <li><b>MI_primary_condition_mapping:</b>\n",
" Maps primary medical condition descriptions to standardized clinical categories for analysis.</li>\n",
" <li><b>MI_secondary_condition_mapping:</b>\n",
" Provides secondary condition-to-category mappings used for comorbidity and risk grouping.</li>\n",
" <li><b>MI_hospitalscores:</b>\n",
" Contains hospital performance and quality scores across clinical, safety, and efficiency domains.</li>"
]
},
{
"cell_type": "markdown",
"id": "62ec1e82-5a40-4b25-b1d7-51b9a9d04c6d",
"metadata": {},
"source": [
"<p style = 'font-size:16px;font-family:Arial'>As we can see from above, we have data in tables about the Procedure, conditions, medications etc. </p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "73eae589-4a6e-4d5a-bcda-82f18f1b4cce",
"metadata": {},
"outputs": [],
"source": [
"df_proc = DataFrame(in_schema(\"DEMO_HealthcareCOC\", \"Va_Procedures\"))\n",
"df_proc"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "51327cef-a8d8-481c-9df8-2b547cbc2804",
"metadata": {},
"outputs": [],
"source": [
"df_proc = DataFrame(in_schema(\"DEMO_HealthcareCOC\", \"Va_Conditions\"))\n",
"df_proc"
]
},
{
"cell_type": "markdown",
"id": "44e70b9d-c9f1-4a17-8b68-9eee7afd83aa",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;\">\n",
"<p style = 'font-size:20px;font-family:Arial'><b>3. Dashboard & Insights </b></p>"
]
},
{
"cell_type": "markdown",
"id": "0b76f833-541f-49bb-b199-e195c06554c7",
"metadata": {},
"source": [
"<p style='font-size:16px;font-family:Arial;color:#00233C'> This section highlights an interactive overview of our Cost of Care Analytics. The dashboard visualizes key metrics related to emergency room utilization, imaging and surgical procedures, and cost drivers across conditions and hospitals. It provides a holistic understanding of how healthcare costs are distributed and where intervention opportunities lie. </p> <p style='font-size:16px;font-family:Arial;color:#00233C'> Key features of the dashboard: <ul style='font-size:16px;font-family:Arial;color:#00233C'> <li>Bar charts showing annual trends in ER visits and costs across multiple categories like Avoidable, Treatable, and Not Preventable conditions.</li> <li>Stacked visualizations depicting proportionate cost contribution by diagnosis and procedure type.</li> <li>Interactive selectors to filter top diagnoses and providers by year, enabling targeted drill-downs.</li> <li>Imaging and surgical trackers to monitor frequency and cost over time for X-rays, MRIs, CT scans, and high-cost surgeries.</li> <li>Geo-mapped insights on hospital performance scores and average procedure costs across counties.</li> </ul> </p>\n",
"\n",
"<p style='font-size:16px;font-family:Arial;color:#00233C'><b>Dashboard Snapshot:</b></p>\n",
"<img src=\"images/cost_of_care.jpg\" style=\"border: 4px solid #404040; border-radius: 10px;\"/>\n",
"\n",
"<p style = 'font-size:16px;font-family:Arial'>\n",
"<div style=\"text-align: left; \">\n",
" <a href=\"https://superset.env.ci.clearscape.teradata.com/superset/dashboard/p/Yv7OP59LyNp/\" style=\"padding: 10px; background-color: #00233c; color: #FFFFFF; text-decoration: none; border-radius: 5px;\">\n",
" Cost Of Care Dashboard\n"
]
},
{
"cell_type": "markdown",
"id": "9bd7325f-73b8-413c-93a1-9151fd66160c",
"metadata": {},
"source": [
"<p style='font-size:16px;font-family:Arial;color:#00233C'>\n",
"The dashboard is structured across multiple tabs to allow detailed investigation:\n",
"</p>\n",
"\n",
"<ul style='font-size:16px;font-family:Arial;color:#00233C'>\n",
" <li><b>Annual-Accross:</b> Tracks ER visits and preventability over time.</li>\n",
" <li><b>Top Diagnoses:</b> Ranks primary and secondary conditions based on cost and frequency.</li>\n",
" <li><b>Xray / Echo Tracker:</b> Flags patients and providers with excessive imaging.</li>\n",
" <li><b>Imaging Studies:</b> Evaluates yearly and monthly volumes and costs of major scans.</li>\n",
" <li><b>Surgical Procedures:</b> Tracks trends and maps surgery cost by region.</li>\n",
" <li><b>Mental Health:</b> Monitors prevalence of selected mental disorders over time.</li>\n",
" <li><b>County Health Rankings:</b> Geospatial overlay of public health indicators across counties.</li>\n",
"</ul>\n",
"\n",
"<p style='font-size:16px;font-family:Arial;color:#00233C'>\n",
"Each tab enables interactive filtering by year, condition, category, and provider to customize the view and drive targeted insights.\n",
"</p>"
]
},
{
"cell_type": "markdown",
"id": "705692a3-a440-4876-a99b-64619920624c",
"metadata": {},
"source": [
"<hr style=\"height:2px;border:none;\">\n",
"<b style = 'font-size:20px;font-family:Arial;'>4. Cleanup</b>"
]
},
{
"cell_type": "markdown",
"id": "a04bf727-bf0c-44fa-8c2f-bea22275aaaa",
"metadata": {},
"source": [
"<p style = 'font-size:18px;font-family:Arial;'> <b>Databases and Tables </b></p>\n",
"<p style = 'font-size:16px;font-family:Arial;'>The following code will clean up tables and databases created above.</p>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "155315f6-0e7f-4aa1-a8c8-76a97338d2aa",
"metadata": {},
"outputs": [],
"source": [
"%run -i ../run_procedure.py \"call remove_data('DEMO_HealthcareCOC');\" # Takes 10 seconds"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9d2086d1-8ca3-4071-8927-156c9a68a7db",
"metadata": {},
"outputs": [],
"source": [
"remove_context()"
]
},
{
"cell_type": "markdown",
"id": "ef410c1e-c64c-4073-9156-1c65aee40e7b",
"metadata": {},
"source": [
"<footer style=\"padding-bottom: 35px; background: #f9f9f9; border-bottom: 3px solid #00233C\">\n",
" <div style=\"float: left; margin-top: 14px;\">ClearScape Analytics™</div>\n",
" <div style=\"float: right;\">\n",
" <div style=\"float: left; margin-top: 14px;\">Copyright © Teradata Corporation - 2025. All Rights Reserved.</div>\n",
" </div>\n",
"</footer>"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.10"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
Binary file added UseCases/CostOfCare_Healthcare/images/cost_of_care.jpg
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.