Data

GR00T-H post-trains GR00T N1.6 on surgical robot data from multiple institutions and robot platforms simultaneously. The core challenge is that each institution records data differently — different robots, coordinate conventions, frame rates, camera setups, and state/action representations. GR00T-H solves this by defining per-embodiment modality configs that convert each dataset into a common representation (REL_XYZ_ROT6D for EEF poses) while preserving robot-specific details like clutch handling and motion scaling. The dataset contains 778 hours of real and synthetic procedure episodes.

An open-source dataset of lung POCUS images derived from a multi-center study involving 226 adult patients presenting to emergency departments with respiratory symptoms. Images were acquired using a standardized scanning protocol (12-zone or modified 8-zone) with various POCUS devices. Videos were preprocessed to remove identifiers, and frames were extracted and standardized to 512×512 pixels using letterboxing to maintain aspect ratios. The dataset contains 1,871 video clips comprising 324,027 frames extracted and standardized to 512×512 pixels. Half of the participants (50%) had COVID-19 pneumonia.

Related publication: Kumar A, Nandakishore P, Gordon AJ, Baum E, Madhok J, Duanmu Y, Kugler J. Creation of an Open-Access Lung POCUS Image Database for Deep Learning and Neural Network Applications. POCUS Journal. 2026;11(1):62-69. 

MedVidBench is a test benchmark for evaluating Video Large Language Models (VLMs) on medical and surgical video understanding. It covers 8 diverse tasks (with GPT-4 and Gemini variants for captioning tasks) across 8 surgical-video datasets. 6,245 test samples across 8 tasks (11 task variants). 8 surgical-video source datasets: AVOS, CholecT50, CholecTrack20, Cholec80_CVS, CoPESD, EgoSurgery, JIGSAWS, NurViD. 103,742 video frames (~18 GB) with per-sample FPS and temporal metadata. Bounding-box annotations for 306 region-caption samples.

Related publication: MedGRPO: Multi-Task Reinforcement Learning for Heterogeneous Medical Video Understanding.Yuhao Su, Anwesa Choudhuri, Zhongpai Gao, et al., arXiv:2512.06581 [cs.CV]

Demo data for 4 tissue regions (959 registered CODEX + H&E + text patches).

Related publications: Linking spatial biology and clinical histology via Haiku. Yan Cui, Jacob S. Leiby, Wenhui Lei, Dokyoon Kim, Yanxiang Deng, Aaron T. Mayer, Zhenqin Wu, Alexandro E. Trevino, Zhi Huang. arXiv:2605.00925 [cs.LG]

This dataset contains Fitbit data from more than 59,000 participants spanning 14 years with more than 39 million step observations and 31 million sleep observations. Nearly half (46%) of participants with Fitbit data also contributed electronic health records, physical measurements, genomics and survey data.

Related publication: Patten, T., Preble, E.A., Master, H. et al. The All of Us Research Program’s wearables dataset. Nat Med (2026). 

This dataset contains data generated from a pipeline using GPT-4o as the LLM. There are 70 patients, each with approximatly 20 – 50 synthetic clinical notes. 50 patients are adults, 20 are paediatrics.

LEMON is a dataset of over 4K surgical high-resolution videos totaling 938 hours from 35 diverse surgical procedure types. Each video is annotated for multi-label classification, indicating the surgical procedures carried out in the video, and for binary classification, indicating if it is robotic or non-robotic. The dataset’s annotations can be found in labels.json.

Related publication: LEMON: A Large Endoscopic MONocular Dataset and Foundation Model for Perception in Surgical Settings. Chengan Che, Chao Wang, Tom Vercauteren, Sophia Tsoka, Luis C. Garcia-Peraza-Herrera. arXiv:2503.19740v4 [cs.CV] 

SafeICU is a freely available database comprising de-identified health-related data from over 2,500 pediatric patients admitted to the pediatric intensive care unit at AIIMS (All India Institute of Medical Sciences), New Delhi, between 2015 and 2025. The AIIMS hosts one of the country’s leading pediatric intensive care units, an independent 8-bed unit that delivers advanced critical care to children with life-threatening illnesses.The SafeICU database includes information such as demographics, bedside vital signs (recorded every 15 seconds), laboratory test results, medications, caregiver notes, microbiology, and mortality. The unit cares for a broad clinical spectrum, with the most frequent reasons for admission being severe respiratory illnesses (such as pneumonia and respiratory failure), sepsis and septic shock, hypertensive crises, congenital heart disease, and acute kidney injury.

Related publication: Akshaya Devadiga, Pradeep Singh, Jhuma Sankar, Rakesh Lodha, & Tavpritesh Sethi(2026). Generalizing intensive care AI across time scales in resource-limited settings. medrxiv.

HealthBench Professional contains 525 physician-authored tasks spanning three clinician-facing use cases: care consult, writing and documentation, and medical research. Each example is designed to evaluate the next model response in a single-turn or multi-turn conversation between a clinician and a model, and is graded via example-specific criteria, similar to HealthBench. HealthBench Professional was built through a process of physician authored annotations with extensive vetting and quality control. A total of 190 physicians contributed to the effort, with practice experience across 50 countries and 26 medical specialties.

Related publication: HealthBench Professional: Evaluating Large Language Models on Real Clinician Chats.Rebecca Soskin Hicks, et al..

This dataset contains a de-identified collection of 100,000 12-lead electrocardiograms (ECGs) with paired structural heart disease (SHD) labels derived from echocardiography, collected at Columbia University Irving Medical Center. Each ECG is provided with raw waveform data sampled at 250 Hz across all 12 leads, along with accompanying demographic and ECG-specific tabular metadata, including age, sex, heart rate, PR interval, QRS duration, and corrected QT interval. Each ECG is annotated with a binary label indicating the presence or absence of structural heart disease based on echocardiographic findings. This dataset was developed as part of the creation of the Columbia Mini-Model, a lightweight deep learning model for SHD detection from ECGs.

Related publication: Hughes, J. W., L. Jing, J. Finer, D. Hartzel, C. Kelsey, A. Long, D. Rocha, J. Ruhl, T. Poterucha and P. Elias “EchoNext-Mini: A Dataset and Baseline AI Model for Detecting Structural Heart Disease from Electrocardiograms.” NEJM AI 0(0): AIdbp2500516.

1 billion rows of genome-wide association study (GWAS) summary statistics from the Psychiatric Genomics Consortium (PGC). Data of  12 psychiatric disorders in this dataset: ADHD, Anxiety, OCD, Mood disorders, PTSD, Schizophrenia, and more…

SurgLaVi is the largest and most diverse surgical vision–language dataset to date, comprising nearly 240k clip–caption pairs from more than 200 procedures, and comprising hierarchical levels at coarse-, mid-, and fine-level. At the core of SurgLaVi lies a fully automated pipeline that systematically generates fine-grained transcriptions of surgical videos and segments them into coherent procedural units. Within this framework, the resulting captions are enriched with contextual detail, producing annotations that are both semantically rich and easy to interpret. To ensure accessibility, the researchers release SurgLaVi-β, an open-source derivative of ~113k clip–caption pairs constructed entirely from public data, which is over four times larger than existing surgical VLP datasets.

Related publication: Perez, A., C. Nwoye, R. Raji Kermani, O. Mohareri and M. Abdullah Jamal (2026). “SurgLaVi: Large-scale hierarchical dataset for surgical vision-language representation learning.” Medical Image Analysis 110: 103982.

Synthetic medical VQA pipeline: 119K images annotated by frontier VLMs, cross-validated at 93% agreement, fine-tuned on 3 model families (2-3B params)

2,966 publications were used for analysis and are made available after final exclusions. The dataset reveals a slight decline in classical Machine Learning (173) and a surge in Multimodal Foundation Models (144) when compared to the previous year in review. Imaging remains the dominant specialty. Data type distribution shows Image data at 53.9% and Text at 38.2%. This dataset tracks the specific trajectories of various specialties, such as Oncology and Surgery, as they adopt higher-capacity foundation models.

Related publication: Edara, R., Khare, A., Atreja, A., Awasthi, R., Highum, B., Hakimzadeh, N., . . . Mathur, P. (2026). Artificial Intelligence in Healthcare: 2025 Year in Review. medRxiv, 2026.2002.2023.26346888. 

This dataset is highly enriched to contain 833 cancer examinations, 1188 biopsy-proven benign examinations, and 3897 examinations flagged as abnormal on screening but subsequently determined to be negative (BI-RADS 1) or benign (BI-RADS 2) at diagnostic imaging.

Related publication: Trivedi, H.M., et al., Open-Source Dataset for the RSNA Screening Mammography Cancer Detection Challenge. Radiology: Artificial Intelligence, 2026. 8(2): p. e250375.

Data consists of clinical test text data (ECGs, CMRs, rest and stress TTEs, ambulatory Holter monitors, CPXs). This study explored the potential of Articulate Medical Intelligence Explorer (AMIE), a large language model-based experimental medical artificial intelligence system, to augment clinical decision-making in this challenging context. We conducted a randomized controlled trial comparing large language model-assisted care with the usual care of complex patients suspected of having a genetic cardiomyopathy, and we curated a real-world dataset of complex cases from a subspecialist cardiology practice.

Related publication: O’Sullivan, J.W., Palepu, A., Saab, K. et al. A large language model for complex cardiology care. Nat Med (2026).

MIMIC-III-Ext-PPG is a large-scale, annotated, and quality-assessed dataset of photoplethysmogram (PPG) segments extracted from the MIMIC-III Waveform Database Matched Subset. It contains 4.9 million non-overlapping 30-second segments from 6,131 critically-ill patients and serves as the largest public resource for heart rhythm classification. The dataset contains signals recorded during several heart rhythms: sinus rhythm, atrial fibrillation, atrial flutter, heart block, pacing, and several less common rhythms.In addition, simultaneous electrocardiogram (ECG, lead II), arterial blood pressure (ABP), and respiratory (RESP) signals are provided where available. Additional annotations were derived from these signals, including blood pressure, heart rate, and respiratory rate.

Related publication: Moulaeifard, M., Charlton, P. H., & Strodthoff, N. (2026). MIMIC-III-Ext-PPG: A PPG Benchmark Dataset for Cardiorespiratory Analysis (version 1.0.0). PhysioNet. RRID:SCR_007345.

This dataset provides eye-tracking data collected while radiologists dictated reports for frontal chest x-rays from the MIMIC-CXR dataset, paired with the timestamped transcription of the dictation. The dataset contains 3,032 cases of synchronized eye-tracking and transcription pairs labeled by five radiologists. For each case, labels for validating algorithms are derived from this dataset. These labels include image-level labels of anomalies found during the reading, ellipses localizing these anomalies, and bounding boxes around lungs and heart. A subset of 109 chest x-rays has readings from all radiologists to estimate the variability of the provided labels.

Related publications:  Bigolin Lanfredi, R., Zhang, M., Auffermann, W., Chan, J., Duong, P., Srikumar, V., Drew, T., Schroeder, J., & Tasdizen, T. (2021). REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays. Sci Data 9, 350 (2022).

VeriFact-BHC contains long-form Brief Hospital Course (BHC) clinical narratives typically found in a discharge summary that have been decomposed into text proposition statements. From 100 patients in the MIMIC-III Clinical Database v1.4, we consider two types of BHC text: a human-written BHC and a LLM-generated BHC. A total of 13,070 propositions are annotated by multiple clinicians with a ground truth established via majority voting and manual adjudication.

Related publication: Chung, Philip, et al. “MIMIC-III-Ext-VeriFact-BHC: Labeled Propositions From Brief Hospital Course Summaries for Long-form Clinical Text Evaluation” (version 1.0.0). PhysioNet (2025). RRID:SCR_007345.

ArchEHR-QA, an expert-annotated dataset of 134 cases from intensive care unit and emergency department settings to evaluate the grounding capabilities of models for responding to patient-initiated queries. The dataset consists of patient-initiated questions posted in public domain, the corresponding clinician-interpreted questions, the excerpts of the EHRs annotated at the sentence-level with relevance to the question, and clinician-generated free-text answers to the questions grounded with EHR sentences. We collect true patient health information needs expressed in real-world health forum messages, we then align the messages to publicly accessible real EHRs. Derived from:MIMIC-III Clinical Database v1.4,MIMIC-IV-Note: Deidentified free-text clinical notes v2.2

Related publication: Soni, S., & Demner-Fushman, D. (2026). ArchEHR-QA: A Dataset for Addressing Patient’s Information Needs related to Clinical Course of Hospitalization (version 1.3). PhysioNet. RRID:SCR_007345.

PARROT is an collaborative initiative to create a multilingual open dataset of radiological reports on which to test LLMs. The aim of PARROT is to represent the diversity of languages and reporting styles to promote applicability of LLM-related research in non-English clinical settings. 2,658 reports, in 14 languages, from 76 authors, from 21 countries.

Related publication: PARROT, an open multilingual radiology reports dataset. Guellec, Bastien Le et al.European Journal of Radiology Artificial Intelligence, Volume 0, Issue 0, 100066

Medical SFT is a curated dataset designed to support the supervised fine-tuning of large language models (LLMs) for medical reasoning tasks. It comprises multi-turn dialogues, clinical case scenarios, and question-answer pairs that reflect the complex reasoning processes encountered in real-world clinical practice.The dataset is intended to help models develop key competencies such as differential diagnosis, evidence-based decision-making, patient communication, and guideline-informed treatment planning. II-Medical SFT is built using a combination of our custom synthetic data generation pipeline and publicly available medical reasoning datasets, ensuring both diversity and clinical relevance. The training dataset comprises 2,197,741 samples.

A high-quality synthetic dataset of synthetic medical reasoning conversations generated using OpenAI’s gpt-oss-120B model with reasoning effort set to high, designed for supervised fine-tuning of large language models in healthcare applications. Intelligent-Internet/II-Medical-Reasoning-SFT used as a seed dataset. Dataset Statistics: Total Samples: 200,927, Total Tokens: 539,165,577,User Messages: 200,847, Assistant Messages: 200,847, Average Tokens per Sample: 2,683.3, Average User Tokens per Sample: 114.1, Average Assistant Tokens per Sample: 2,569.2.  Each conversation demonstrates structured medical thinking with step-by-step reasoning processes.

The Cancer Genome Atlas (TCGA) Multimodal Dataset is a comprehensive collection of clinical data, pathology reports, slide images, molecular data, and radiology images for cancer patients. This dataset aims to facilitate research in multimodal machine learning for oncology by providing embeddings generated using state-of-the-art models including GatorTron, MedGemma, Qwen, Llama, UNI, SeNMo, REMEDIS, and RadImageNet. 10,771 patient records per model for clinical data and 10,857 patient records per model for pathology reports.

Related publication: Tripathi, A., Waqas, A., Schabath, M.B. et al. HONeYBEE: enabling scalable multimodal AI in oncology through foundation model-driven embeddings. npj Digit. Med.8, 622 (2025).

A comprehensive synthetic healthcare dataset containing 575,415 patients with complete medical histories, generated using Synthea – an open-source synthetic patient generator. Generated by Richard Young, DeepNeuro.AI.

Related publications: Walonoski J, Kramer M, Nichols J, Quina A, Moesel C, Hall D, Duffett C, Dube K, Gallagher T, McLachlan S. Synthea: An approach, method, and software mechanism for generating synthetic patients and the synthetic electronic health care record. J Am Med Inform Assoc. 2018 Mar 1;25(3):230-238.

STARC-9 (STAnford coloRectal Cancer): A large-scale dataset with 630k samples across nine tissue types (~70k per class) collected from over 200 WSI for multi-class tissue classification task. Stanford and TCGA-CRC tile-level validation datasets.Pretrained model weights on STARC-9, including baseline, SOTA, and Pathology-specific foundation models.Open-source code repository for DeepCluster++ framework that can be used for image sample collection across domains.

Related publication: STARC-9: A Large-scale Dataset for Multi-Class Tissue Classification for CRC Histopathology. Barathi Subramanian, et al. https://arxiv.org/abs/2511.00383

Clinical risk prediction based on machine learning algorithms plays a vital role in modern healthcare. A crucial component in developing a reliable prediction model is a high-quality dataset with time series clinical events. In this work, the authors release such a dataset that consists of 22,588,586 clinical time series events, which is term MIMIC-IV-Ext-22MCTS. The source data are discharge summaries selected from the well-known yet unstructured MIMIC-IV-Note. The authors then extract clinical events as short text spans from the discharge summaries, along with the timestamps of these events as temporal information by contextual retrieval and Llama-3.1-8B.

Related publication: Wang, J., Niu, X., Zhang, T., Shen, J., Kim, J., & Weiss, J. (2025). MIMIC-IV-Ext-22MCTS: A 22 Millions-Event Temporal Clinical Time-Series Dataset with Relative Timestamp (version 1.0.0). PhysioNet. RRID:SCR_007345.

A collection of resources that provide accelerometer data. This is a compilation of datasets from Mobilize center  group at Stanford University and others. Key datasets listed are the following: AddBiomechanics Dataset, Understanding Walking in Parkinson’s Disease with Wearable Sensors,  Daphnet Freezing of Gait Data Set, The Osteoarthritis Initiative (OAI) dataset, The National Health and Nutrition Examination Survey (NHANES), Human Activity Recognition Using Smartphones Data Set, ADL Recognition with Wrist-worn Accelerometer Data Set, Activity Recognition from Single Chest-Mounted Accelerometer Data Set, International Children’s Accelerometry Database (ICAD), UK Biobank, Avon Longitudinal Study of Parents and Children, Multicenter Osteoarthritis Study (MOST), MHEALTH Dataset

This dataset contains data from 2280 participants that was collected between July 19, 2023 and May 01, 2025. Data from multiple modalities are included. The AI-READI is a dataset consisting of data collected from individuals with and without Type 2 Diabetes Mellitus (T2DM) and harmonized across 3 data collection sites. The composition of the dataset was designed with future studies using AI/Machine Learning in mind. This included recruitment sampling procedures aimed at achieving approximately equal distribution of participants across diabetes severity, as well as the design of a data acquisition protocol across multiple domains (survey data, physical measurements, clinical data, imaging data, wearable device data, etc.) to enable downstream AI/ML analyses that may not be feasible with existing data sources such as claims or electronic health records data. The dataset contains 358,999 files and is around 3.87 TB in size.

Related publication: AI-READI Consortium. (2024). “AI-READI: rethinking data collection, preparation and sharing for propelling AI-based discoveries in diabetes research and beyond.”
Nature metabolism. 

HC4 is a large-scale pretraining dataset containing over 65 billion tokens from diverse healthcare-related sources. The corpus was curated to enable systematic investigation of how data composition influences language model behavior, including potential demographic biases. 153GB (around 65 billion tokens). 9.7+ million documents from diverse sources including peer-reviewed scientific literature collected from PubMed Central, Semantic Scholar, OpenAlex repositories.

Related publication: Building Trust in Clinical LLMs: Bias Analysis and Dataset Transparency (EMNLP 2025). https://arxiv.org/pdf/2510.18556

The first manually annotated, publicly available dataset that links free-text radiology findings to 3D segmentations in chest CT scans. The dataset contains:
• 3,142 non-contrast chest CT scans from CT-RATE• 8,028 findings with 16,301 entities across focal and non-focal patterns • Annotations monitored through a two-tiered QA process, with validation/test sets fully labeled by board-certified radiologists• Includes an Anatomical Chain-of-Thought (CoT) dataset, capturing radiologist-style step-by-step reasoning for localizing findings.

Related publication: ReXGroundingCT: A 3D Chest CT Dataset for Segmentation of Findings from Free-Text Reports. https://arxiv.org/abs/2507.22030v2

The OhioT1DM dataset is available to researchers interested in improving the health and wellbeing of people with type 1 diabetes. It contains 8 weeks worth of data for each of 12 people with type 1 diabetes. These people were all on insulin pump therapy with continuous glucose monitoring (CGM). They provided blood glucose data, insulin data, self-reported life-event data, and data from physiological fitness bands. The dataset includes: a CGM blood glucose level every 5 minutes; blood glucose levels from periodic self-monitoring of blood glucose (finger sticks); insulin doses, both bolus and basal; self-reported meal times with carbohydrate estimates; self-reported times of exercise, sleep, work, stress, and illness; and physiological data from fitness bands.

Related publication: The OhioT1DM Dataset for Blood Glucose Level Prediction: Update 2020

MIDRC is a multi-institutional collaborative initiative driven by the medical imaging community that was initiated in late summer 2020. It has more than 500,000 imaging studies,representing more than 80,000 cases,  with around 200,000 available for public use. The data in MIDRC are open access in order to foster machine learning innovation through data sharing and include in addition to imaging files, patient demographic data, COVID-19 test results and other clinical data, harmonized study descriptions utilizing the LOINC playbook, and image DICOM tags for purposes of data filtering and selecting cohorts for analysis.

The dataset consists of 1,255 cervical spine magnetic resonance imaging (MRI) examinations from 1,232 patients collected from the Duke University Health System. CSpineSeg also includes expert manual semantic segmentations of vertebral bodies and intervertebral discs for 481 patients.

Related publication: Zhou, L., Wiggins, W., Zhang, J. et al. The Duke University Cervical Spine MRI Segmentation Dataset (CSpineSeg). Sci Data 12, 1695 (2025).

A multilingual dataset of 477 wound care cases, 768 responses, 748 images, 3k structured data labels, 1362 translation instances, and 10k judgments.

Related publication: Yim WW, rt al. WoundcareVQA: A multilingual visual question answering benchmark dataset for wound care. J Biomed Inform. 2025 Oct;170:104888.

GPTNERMED is a novel open synthesized dataset and neural named-entity-recognition (NER) model for German texts in medical natural language processing (NLP).This dataset contains the synthetic German sentences with annotated entities (Medikation, Dosis, Diagnose) from the GPTNERMED project. The sentences as well as the annotations are not manually validated by medical professionals and therefore this dataset is no gold standard dataset.The dataset consists of 9,845 sentences (121,027 tokens by SpaCy Tokenizer, 245,107 tokens by the GPT tokenizer).

Related publication: Frei J, Kramer F. Annotated dataset creation through large language models for non-english medical NLP. J Biomed Inform. 2023 Sep;145:104478. 

MedPix 2.0 is derived from a freely open-access source MedPix. It offers a balanced variety of CT and MRI scans of different body parts.For each image, a complete structured clinical case is provided. MedPix^® is a free open-access multimodal online database of medical images, teaching cases, and clinical topics, managed by the National Library of Medicine (NLM) of the National Institutes of Health (NIH). It mainly serves as a support system for Continuing Medical Education (CME) of physicians, nurses, and healthcare students. The database collects clinical cases related to more than 12,000 patients. Each case contains at least one medical image, and the corresponding findings, discussion notes, diagnosis, differential diagnosis, treatment, and follow up. Textual information is reported in a semi-structured format. Attached to the clinical case, there is the topic section, where the disease under investigation is discussed in detail from an academic and general perspective.

Related publication: Siragusa, I., Contino, S., Ciura, M.L. et al. MedPix 2.0: A Comprehensive Multimodal Biomedical Data Set for Advanced AI Applications with Retrieval Augmented Generation and Knowledge Graphs. Data Sci. Eng. (2025). 

Large-scale medical dataset. FFA-IR collects 766 reports along with 47,247 FFA images from clinical practice. Explainable annotation. FFA-IR annotates 46 categories of lesions with a total of 12,166 regions. Bilingual reports. FFA-IR provides both English and Chinese reports for each case.

Related publication: Li, M., et al. FFA-IR: Towards an Explainable and Reliable Medical Report Generation Benchmark (version 1.1.0). PhysioNet. RRID:SCR_007345. 

239,025 additional X-ray image-text pairs from MIMIC-CXR using GPT-4, expanding the dataset to 400,042 pairs – more than doubling its original size.

Related publication: Towards a clinically accessible radiology foundation model: open-access and lightweight, with automated evaluation. Juan Manuel Zambrano Chaves, et al. https://arxiv.org/abs/2403.08002

Features 300 physician-written tasks across 10 medical categories, covering essential clinical workflows. Built on a FHIR-compliant interactive setup with 100 de-identified realistic patient profiles and 700,000+ medical data points, enabling practical and scalable AI testing.

Related publication: MedAgentBench: A Realistic Virtual EHR Environment to Benchmark Medical LLM Agents. Yixing Jiang, et al. https://arxiv.org/abs/2501.14654

The Mobile Brazilian Retinal Dataset (mBRSET) is a collection of retinal images captured via portable cameras, encompassing a diverse range of ethnic backgrounds from Itabuna, Bahia, Brazil. Comprising 5164 images from 1291 patients diagnosed with diabetes, this dataset is augmented with clinical and demographic metadata.

Related publication: Nakayama, L. F., et al. (2024). mBRSET, a Mobile Brazilian Retinal Dataset (version 1.0). PhysioNet. RRID:SCR_007345. https://doi.org/10.13026/qxpd-1y65

ECG-Grounding provides more accurate, holistic, and evidence-driven interpretations with diagnoses grounded in measurable ECG features. Currently, it contains 30,000 instruction pairs annotated with heartbeat-level physiological features. This is the first high-granularity ECG grounding dataset, enabling evidence-based diagnosis and improving the trustworthiness of medical AI.

Related publication: GEM: Empowering MLLM for Grounded ECG Understanding with Time Series and Images. Xiang Lan, et al. https://arxiv.org/abs/2503.06073

An open-source dataset derived from the PubMed Central Open Access subset, containing over 6 million scientific articles and 24 million image-text pairs, along with 27 metadata fields (including expert human annotations).

Related publication: A Large-Scale Vision-Language Dataset Derived from Open Scientific Literature to Advance Biomedical Generalist AI. Alejandro Lozano, et al. https://arxiv.org/abs/2503.22727

1506 expert-validated 3D segmentations allow for the development of generalizable and robust AI models for breast cancer detection and segmentation. Pre-trained nnU-Net weights are included to serve as a baseline.

Related publication: Garrucho, L., Kushibar, K., Reidel, CA. et al. A large-scale multicenter breast cancer DCE-MRI benchmark dataset with expert segmentations. Sci Data12, 453 (2025)

This dataset comprises 628 abdominal MRI scans (310 T1-weighted (T1W) and 318 T2-weighted (T2W)) volumetric scans along with corresponding segmentation masks annotated by physicians. CirrMRI600+ is a single-center, multivendor, and multisequence dataset. CirrMRI600+ is the first dataset designed explicitly for liver cirrhosis research and incorporates both T1W and T2W MRI images.

A diverse database of more than 500 datasets including OMOP standardized electronic medical records, surgical patient datasets, EKG dataset, Clinical notes, anesthesia records,chest X-rays, cancer registry, Knee X-ray and more. from Seoul National University Hospital(SNUH)

All input data and associated expert responses for all 857 case studies in sleep and fitness used for training and evaluation, along with all model responses to the 100 holdout case studies, are stored in JSON format.All input and target prediction data, which include bucketized age, daily sensor readings for 20 sensors over 15 days and 16 binary survey responses, for all 4,163 samples in the PRO task are stored in JSON format.

Related publication: Khasentino, J., Belyaeva, A., Liu, X. et al. A personal health large language model for sleep and fitness coaching. Nat Med (2025).

FactEHR is a benchmark dataset designed to evaluate the ability of large language models (LLMs) to perform factual reasoning over clinical notes. It includes:

• 2,168 deidentified notes from multiple publicly available datasets
• 8,665 LLM-generated fact decompositions
• 987,266 entailment pairs evaluating precision and recall of facts
• 1,036 expert-annotated examples for evaluation

FactEHR supports LLM evaluation across tasks like information extraction, entailment classification, and model-as-a-judge reasoning.

Related publication: FactEHR: A Dataset for Evaluating Factuality in Clinical Notes Using LLMs. Monica Munnangi, et al. https://doi.org/10.48550/arXiv.2412.12422.

A large-scale synthetic instruction dataset of 2.5M rows (~700k unique instructions) for clinical natural language processing covering 14 task types and 98 fine-grained input clinical documents.

Related publication: A Modular Approach for Clinical SLMs Driven by Synthetic Data with Pre-Instruction Tuning, Model Merging, and Clinical-Tasks Alignment. Jean-Philippe Corbeil, Amin Dada, Paul Vozila, et al. arXiv:2505.10717v2

A public bilingual dataset of 4,555 CXR studies with substantiated reports (3,099 abnormal and 1,456 normal), each containing complete lists of sentences describing individual positive and negative findings in English and Spanish. In total, PadChest-GR contains 7,037 sentences of positive findings and 3,422 sentences of negative findings. This dataset is derived from PadChest dataset.

Related publication: Castro DCd, Bustos A, Bannur S, et al. PadChest-GR: A Bilingual Chest X-Ray Dataset for Grounded Radiology Report Generation. NEJM AI. 2025;2(7):AIdbp2401120.

The survey was administered to a nonprobability convenience sample at 74 COMFORT network hospitals across 43 countries.In total, 13 806 patients participated, including 8951 (64.8%) in the Global North and 4855 (35.2%) in the Global South. Their median (IQR) age was 48 (34-62) years, and 6973 (50.5%) were male. The survey instrument consisted of 3 dimensions: trust in AI, AI and diagnosis, and preferences and concerns toward AI, each with 6 items, complemented by a general data section with 8 items (self-reported gender, age, highest educational level, weekly use of technological devices, health status, AI knowledge, and general attitudes toward AI use in medicine and health care).

Related publication: Busch F, Hoffmann L, Xu L, et al. Multinational Attitudes Toward AI in Health Care and Diagnostics Among Hospital Patients. JAMA Netw Open. 2025;8(6):e2514452. doi:10.1001/jamanetworkopen.2025.14452

This study examined the relationships between coping styles, stress levels, and pain levels in lung cancer patients, and whether these relationships were moderated by place of residence (small town vs large city). A sample of 97 lung cancer patients completed questionnaires measuring coping styles (Stress Coping Inventory), perceived stress levels (Perceived Stress Scale), and pain levels (Short Form McGill Pain Questionnaire).

A dataset based on PubMed search of 28,180 total articles; 1,693 were initially labeled mature, after which 1,551 articles were analyzed after exclusions.The most prevalent specialties within our PubMed search originated from imaging (407), head and neck (127), and General (122). Analysis of AI model types showed that the Large Language Model (LLM) was the most popular utilized in 479 publications, followed by AI General (448), and DL (372). Qualitative data was obtained on the data types, and it was revealed that the image data was predominant and used in 57.0% of the mature sources, followed by text (33.1%), followed by tabular (7.59%). The utilization of Large Language Models (LLMs) is the highest in publications associated with education at 18.6%, followed by General at 13.6%.

Related publication: Artificial Intelligence in Healthcare: 2024 Year in Review. Raghav Awasthi, et al. medrxiv.org/10.1101/2025.02.26.25322978v2

The Hillel Yaffe Glaucoma Dataset (HYGD) addresses a critical limitation in existing GON datasets: the lack of gold-standard annotations. The dataset was curated by the Hillel Yaffe Ophthalmology Department Glaucoma Unit, Hadera, Israel, between 2022-2024. The dataset consists of 747 DFIs, including:548 glaucomatous DFIs (73%) and 199 non-glaucomatous DFIs (27%). HYGD’s labels are based on comprehensive ophthalmic examinations, including visual acuity assessment, intraocular pressure measurement, optical coherence tomography (OCT), visual field tests, and at least one year of follow-up monitoring.The dataset is structured to include both the DFIs and a labels file containing patient IDs, GON classifications and image quality scores.

Related publication: Abramovich, O., Pizem, H., Fhima, J., Berkowitz, E., Gofrit, B., Van Eijgen, J., Blumenthal, E., & Behar, J. (2025). Hillel Yaffe Glaucoma Dataset (HYGD): A Gold-Standard Annotated Fundus Dataset for Glaucoma Detection (version 1.0.0). PhysioNet. RRID:SCR_007345.

Bridge2AI-Voice v2.0 contains data for 19,271 recordings collected from 442 participants across five sites in North America. Participants were selected based on known conditions which manifest within the voice waveform including voice disorders, neurological disorders, mood disorders, and respiratory disorders.

Related publication: Bensoussan, Y.,  et al.  (2025). Bridge2AI-Voice: An ethically-sourced, diverse voice dataset linked to health information (version 2.0.0). PhysioNet. 

The BRIDGE-Open dataset is an open-access subset of the datasets used by the BRIDGE benchmark. This dataset includes the 55 tasks of the open-access datasets that can be released to the public.  Due to privacy and security considerations of clinical data, regulated-access datasets can not be directly published. However, all detailed task descriptions and their corresponding data sources are available in the BRIDGE paper, Supplementary Section 5. Importantly, all 87 datasets have been verified to be either fully open-access or publicly accessible via reasonable request in the systematic review.

Related publication: BRIDGE: Benchmarking Large Language Models for Understanding Real-world Clinical Practice Text. Jiageng Wu, et al. https://arxiv.org/abs/2504.19467

The largest annotated lung ultrasound (LUS) repositories to date—over 300,000 de-identified frames, drawn from 226 patients with diverse respiratory diagnoses. Includes both COVID and non-COVID pneumonia, pulmonary edema, COPD, healthy controls, and more.

Related publication: Creation of an Open-Access Lung Ultrasound Image Database For Deep Learning and Neural Network Applications. Andre Kumar, et al. medRxiv 2025.05.09.25327337

This file contains the dataset. It has 24 diseases each with 50 symptoms giving a total of 1200 datapoints.

The Collab-CXR dataset is generated from an experiment first analyzed and reported in⁶, which collects chest X-ray diagnostic assessments for retrospective patient cases of 227 radiologists for 104 pathologies using a remote interface designed for this study.The dataset contains additional information on the time spent by radiologists on each case and the values of two diagnostic standards defined by the researchers.This dataset is the largest in the current literature on AI for radiology that can be used to compare human-AI collaboration with AI alone or humans alone.

Related publication: Moehring, A., Kutwal, M., Huang, R. et al. A Dataset for Understanding Radiologist-Artificial Intelligence Collaboration. Sci Data12, 739 (2025). https://doi.org/10.1038/s41597-025-05054-0

OpenBiomedVid is a biomedical video instruction tuning dataset comprising 1031 hours of video-caption and Q/A pairs, curated through a multi-step human-in-the-loop pipeline. Includes two new expert-curated biomedical video benchmarks, SurgeryVideoQA and MIMICEchoQA, consisting of 2,692 and 622 video-QA pairs respectively. SurgeryVideoQA is curated from relevant and high-quality surgical videos from YouTube, while MIMICEchoQA is derived from the MIMIC-IV-ECHO (Gow et al., 2023) and focuses on echocardiography videos paired with clinical questions.

Related publication: How Well Can General Vision-Language Models Learn Medicine By Watching Public Educational Videos? Rahul Thapa, Andrew Li, David Ouyang, James Zou, et al. https://doi.org/10.48550/arXiv.2504.14391

Tahoe-100M is a giga-scale single-cell perturbation atlas consisting of over 100 million transcriptomic profiles from 50 cancer cell lines exposed to 1,100 small-molecule perturbations. Generated using Vevo Therapeutics’ Mosaic high-throughput platform, Tahoe-100M enables deep, context-aware exploration of gene function, cellular states, and drug responses at unprecedented scale and resolution. This dataset is designed to power the development of next-generation AI models of cell biology, offering broad applications across systems biology, drug discovery, and precision medicine.

Related publication: Tahoe-100M: A Giga-Scale Single-Cell Perturbation Atlas for Context-Dependent Gene Function and Cellular Modeling

A publicly available research dataset in perioperative medicine, which includes approximately 130,000 cases (50% of all surgical cases) who underwent anesthesia for surgery at an academic institution in South Korea between 2011 and 2020. This comprehensive dataset includes patient characteristics such as age, sex, American Society of Anesthesiologists physical status classification, diagnosis, surgical procedure code, department, and type of anesthesia. It also includes vital signs in the operating theatre, general wards, and intensive care units (ICUs), laboratory results from six months before admission to six months after discharge, and medication during hospitalization. Complications include total hospital and ICU length of stay and in-hospital death.

Related publication: Lim, L., Lee, H., Jung, CW. et al. INSPIRE, a publicly available research dataset for perioperative medicine. Sci Data11, 655 (2024). https://doi.org/10.1038/s41597-024-03517-4

MOVER is a repository containing data associated with hospital visits for patients undergoing surgery at the University of California, Irvine (UCI) Medical Center in Orange County, California.  The data included in MOVER was collected over 4 years and contains comprehensive EMR records and waveforms for each patient encounter. These records include general information about each patient and their medical history, and specific information regarding the the surgical procedure being performed including: medicines used, lines or drains used, and post-operative complications. The repository includes 58,799 unique patients with data from 83,468 surgeries. MOVER is freely available for download for all researches who sign a data usage agreement and is intended to advance a wide variety of healthcare research.

Related publication: Samad M, Angel M, Rinehart J, Kanomata Y, Baldi P, Cannesson M. Medical Informatics Operating Room Vitals and Events Repository (MOVER): a public-access operating room database. JAMIA Open. 2023 Oct 17;6(4):ooad084. doi: 10.1093/jamiaopen/ooad084. PMID: 37860605; PMCID: PMC10582520.

Dataset for summarization of long documents, using Pubmed articles summarized with abstracts. The dataset has 3 splits: train, validation, and test. Train data with 119,924 instances, validation data with 6,633 instances and the Test data with 6,658 instances.

Chest X-Ray Report Errors (ReXErr-v1), a new dataset based on MIMIC-CXR and constructed using large language models (LLMs) that contains synthetic error reports for the majority of MIMIC-CXR. Developed with input from board-certified radiologists, ReXErr-v1 contains plausible errors that closely mimic those found in real-world scenarios. Furthermore, ReXErr-v1 utilizes a novel sampling methodology that selects three errors to inject among a set of frequent errors made by both human and AI models. The dataset includes errors both at report and sentence level.

Related publication:  Rao, V. M., Zhang, S., Acosta, J. N., Adithan, S., & Rajpurkar, P. (2024). ReXErr: Synthesizing Clinically Meaningful Errors in Diagnostic Radiology Reports. In Biocomputing 2025: Proceedings of the Pacific Symposium (pp. 70-81).

The Pathology Images of Scanners and Mobilephones (PLISM) dataset was created for the evaluation of AI models’ robustness to domain shifts. PLISM is the first group-wised pathological image dataset that encompasses diverse tissue types stained under 13 H&E conditions, with multiple imaging media, including smartphones (7 scanners and 6 smartphones).

• In PLISM-sm, smartphone images were used as queries to create image groups for each staining condition corresponding to each tile image. The PLISM-sm subset contains a total of 57,902 images.
• In PLISM-wsi, smartphone images were used as queries to create image groups for each staining condition corresponding to each tile image. The PLISM-wsi subset contains a total of 310,947 images
• Color and texture in digital pathology images are affected by H&E stain conditions (e.g. Harris or Carrazi) and digitalization devices (e.g. slide scanners or smartphones), which cause inter-institutional domain shifts.

118,385 emergency department visits with continuous physiological waveforms.

Multimodal Clinical Monitoring in the Emergency Department (MC-MED), a comprehensive, multimodal, and de-identified clinical and physiological dataset. MC-MED includes 118,385 adult ED visits to an academic medical center from 2020 to 2022. Data include continuously monitored vital signs, physiologic waveforms (electrocardiogram, photoplethysmogram, respiration), patient demographics, medical histories, orders, medication administrations, laboratory and imaging results, and visit outcomes. MC-MED is the first dataset to combine detailed physiologic monitoring with clinical events and outcomes for a large, diverse ED population.

The dataset consists of three different partitions: DiagSet-A, containing over 2.6 million tissue patches extracted from 430 fully annotated scans; DiagSet-B, containing 4675 scans with assigned binary diagnosis; and DiagSet-C, containing 46 scans with diagnosis given independently by a group of histopathologists.

Related Publication: Koziarski, M., Cyganek, B., Niedziela, P. et al. DiagSet: a dataset for prostate cancer histopathological image classification. Sci Rep14, 6780 (2024). https://doi.org/10.1038/s41598-024-52183-4

It includes 3,848 clinical texts from the MS and University of Washington hospital collections covering five types of errors (Diagnosis, Management, Treatment, Pharmacotherapy, and Causal Organism).

Related publication: MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes. arXiv:2412.19260v2 [cs.CL] 2 Jan 2025

BiomedParseData by combining 45 biomedical image segmentation datasets and using GPT-4 to generate the canonical semantic label for each segmented object.  GPT-4 was used to create a unifying biomedical object ontology for image analysis and harmonize natural language descriptions with this ontology. This ontology encompasses three main categories (histology, organ and abnormality), 15 meta-object types and 82 specific object types. The resulting BiomedParseData contains 3.4 million distinct image–mask–label triples, spanning nine imaging modalities and 25 anatomic sites representing a large-scale and diverse dataset for semantic-based biomedical image analysis.The images include CT scan, MRI,Chest X-ray, ultrasound, skin lesion photos,Endoscopy images, Pathology whole slide images, and eye OCTs.

Related publication: Zhao, T., Gu, Y., Yang, J. et al. A foundation model for joint segmentation, detection and recognition of biomedical objects across nine modalities. Nat Methods (2024). 

A unified dataset of 2,418 FDA-approved drugs with drug-induced toxicity summaries and ratings created by using GPT-4o to process FDA drug labels. UniTox spans eight types of toxicity: cardiotoxicity, liver toxicity, renal toxicity, pulmonary toxicity, hematological toxicity, dermatological toxicity, ototoxicity, and infertility.

Related publication: UniTox:Leveraging LLMs to Curate a Unified Dataset of Drug-Induced Toxicity from FDA Labels.  Jake Silberg et al., https://doi.org/10.1101/2024.06.21.24309315. Medrxiv

An open source benchmark for medical concepts question answering. The benchmark comprises of questions of various medical concepts across different vocabularies: diagnoses, procedures, and drugs. The questions are categorized into three levels of difficulty: easy, medium, and hard.

Related publication: Ofir Ben Shoham, Nadav Rappoport,MedConceptsQA: Open source medical concepts QA benchmark,Computers in Biology and Medicine,Volume 182,2024,109089.

The following is an information retrieval test collection that contains: * 204,855 publicly available clinical trails was crawled from ClinicalTrials.gov. * 60 topics made up of three types: patient case descriptions, patient case summaries and assessor provided ad-hoc queries, totalling an average of 10.2 queries per topic. * 4,000 assessor provided relevance assessment for topic, trial pairs.

Related publication:  Bevan Koopman and Guido Zuccon. 2016. A Test Collection for Matching Patients to Clinical Trials. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval (SIGIR ’16). Association for Computing Machinery, New York, NY, USA, 669–672. https://doi.org/10.1145/2911451.2914672

1. TREC Genomics (2003-2007). This track focused on genomics researchers seeking relevant biomedical literature.
2. TREC Medical Records (2011-2012). This track focused on retrieving cohorts of patients from electronic health records (EHRs).
3. Clinical Decision Support (2014, 2015, 2016). This track focused on clinicians looking for evidence-based full-text literature to support diagnosis, treatment, and testing decisions.
4. Precision Medicine (2017, 2018, 2019, 2020). This track focused on oncologists looking for evidence-based treatment literature and clinical trials.
5. Clinical Trials (2021, 2022, 2023). This (ongoing) track focuses on matching patients to relevant clinical trials.

The dataset is a collection of retrospective pre-operative brain magnetic resonance imaging (MRI) scans, clinically acquired from six diagnostic centers in Nigeria. The scans are from 146 patients who have brain MRIs indicating central nervous system neoplasms, diffuse glioma, low-grade glioma, or glioblastoma/high-grade glioma. The brain scans were multiparametric MR images (mpMRI), specifically T1, T1 CE, T2, and T2 FLAIR,  acquired on 1.5T MRI between January 2010 and December 2022.The expert-annotated tumor sub-regions for each of the 146 cases are provided along with a metadata (csv file) of study location, scanner type, where available.

The MTS-Dialog dataset is a new collection of 1.7k short doctor-patient conversations and corresponding summaries (section headers and contents).The training set consists of 1,201 pairs of conversations and associated summaries.The validation set consists of 100 pairs of conversations and their summaries.MTS-Dialog includes 2 test sets; each test set consists of 200 conversations and associated section headers and contents.

The augmented dataset consists of 3.6k pairs of medical conversations and associated summaries created from the original 1.2k training pairs via back-translation using two languages French and Spanish, as described in the paper.

Related publication: Asma Ben Abacha, Wen-wai Yim, Yadan Fan, and Thomas Lin. 2023. An Empirical Study of Clinical Note Generation from Doctor-Patient Encounters. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2291–2302, Dubrovnik, Croatia. Association for Computational Linguistics.

The corpus, created from domain experts, is designed to model three variations of model-assisted clinical note generation from doctor-patient conversations. These include conversations with (a) calls to a virtual assistant (e.g. required use of wake words or prefabricated, canned phrases), (b) unconstrained directions or discussions with a scribe, and (c) natural conversations between a doctor and patient. Contains 1342 samples.

Related publication: Yim, Ww., Fu, Y., Ben Abacha, A. et al. Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation. Sci Data 10, 586 (2023). https://doi.org/10.1038/s41597-023-02487-3

The UltraMedical Collections is a large-scale, high-quality dataset of biomedical instructions, comprising 410,000 synthetic and manually curated samples.

Related publication: UltraMedical: Building Specialized Generalists in Biomedicine. Kaiyan Zhang, Sihang Zeng, Bowen Zhou, et al. arXiv:2406.03949v1

RadGraph2 is a dataset of 800 chest radiology reports annotated using a fine-grained entity-relationship schema, which is an expanded version of the previously introduced RadGraph dataset. RadGraph2,  focuses on capturing changes in disease state and device placement over time. It introduces a hierarchical schema that organizes entities based on their relationships and show that using this hierarchy during training improves the performance of an information extraction model. In addition to the dataset of manually labeled reports, we release more than 220,000 reports automatically annotated by our benchmark model. This model achieved an F1 micro performance of 0.88 and 0.74 on two differently sourced withheld test sets (from MIMIC-CXR-JPG and CheXpert, respectively).

Related publication: RadGraph2: Modeling Disease Progression in Radiology Reports via Hierarchical Information Extraction. Sameer Khanna, Adam Dejl, Pranav Rajpurkar et al. Proceedings of the 8th Machine Learning for Healthcare Conference, PMLR 219:381-402, 2023.

A total of 167 skin tone variables and two temperature variables are collected per body location, excluding images, together with ten non-biometric body location images per patient and the associated electronic health record (EHR) data. The ENCoDE project is a comprehensive EHR-linked skin tone database to combat skin tone associate disparities.The study included patients admitted to Duke University Hospital with pulse oximetry recorded up to 5 minutes prior to arterial blood gas (ABG) measurements. Skin tone was measured across sixteen body locations using administered visual scales (Fitzpatrick, Monk Skin Tone, and Von Luschan), reflectance colorimetry (Delfin SkinColorCatch), and reflectance spectrophotometry (Konica Minolta CM-700D, Variable Spectro 1). IPhone SE 2020 and Google Pixel 4 (Android) image data are available for non-biometric body locations.

Github: https://physionet.org/content/encode-skin-color/1.0.0/

NCHS provides updated US healthcare related datasets based on surveys and reported data.

The largest dataset to date tackling the problem of AI-assisted note generation from visit dialogue. We also present the benchmark performances of several common state-of-the-art approaches.A corpus of textual data corresponding to synthetic clinical encounters, including each encounters’ dialogue transcript and clinical notes.

Related publication: Yim, Ww., Fu, Y., Ben Abacha, A. et al. Aci-bench: a Novel Ambient Clinical Intelligence Dataset for Benchmarking Automatic Visit Note Generation. Sci Data10, 586 (2023). https://doi.org/10.1038/s41597-023-02487-3

The largest trial outcome dataset with around 479K clinical trials, aggregating outcomes from multiple sources of weakly supervised labels, minimizing the noise from individual sources, and eliminating the need for human annotation.

Related publication: Automatically Labeling $200B Life-Saving Datasets: A Large Clinical Trial Outcome Benchmark. Chufan Gao, Jathurshan Pradeepkumar, Trisha Das, Shivashankar Thati, Jimeng Sun. arXiv:2406.10292 [cs.AI]

The World Health Organization manages and maintains a wide range of data collections related to global health and well-being as mandated by our Member States.

The CheXpert Plus dataset is a comprehensive collection that brings together text and images in the medical field, featuring a total of 223,462 unique pairs of radiology reports and chest X-rays across 187,711 studies from 64,725 patients.The X-rays are provided in DICOM format, including 47 DICOM metadata elements to support detailed analysis. Accompanying these images are 187,711 radiology reports, each meticulously divided into 11 subsections for thorough examination. Finally, the dataset is enriched with annotations for 14 different chest pathologies across the studies, alongside 8 metadata elements concerning patient information.

Related publication: CheXpert Plus: Augmenting a Large Chest X-ray Dataset with Text Radiology Reports, Patient Demographics and Additional Image Formats. Pierre Chambon, Jean-Benoit Delbrouck, , Curtis P. Langlotz, et al. arXiv:2405.19538

RadEvalX focuses on radiologist evaluations of errors found in automatically generated radiology reports. The dataset includes annotations from two board-certified radiologists, who identified clinically significant and clinically insignificant errors across eight different categories of errors. A balanced dataset of 100 reports for human annotation from an initial set of 590 reports generated using M2Tr

Related publications: Calamida, A. R., Nooralahzadeh, F., Rohanian, M., Nishio, M., Fujimoto, K., & Krauthammer, M. (2024). Radiology Report Generation Models Evaluation Dataset For Chest X-rays (RadEvalX) (version 1.0.0). PhysioNet. 

2 Sample datasets for histopathology foundation model Prov-GigaPath.

Sample 1: https://zenodo.org/records/10909616

Sample 2: https://zenodo.org/records/10909922

Related publication: Xu, H., Usuyama, N., Bagga, J. et al. A whole-slide foundation model for digital pathology from real-world data. Nature (2024). https://doi.org/10.1038/s41586-024-07441-w

DECIPHER is used by the clinical community to share and compare phenotypic and genotypic data. The DECIPHER database contains data from 48,774 patients who have given consent for broad data-sharing.

Genomic and phenotypic data are made accessible to the scientific community through dbGaP, with new data submissions occurring annually.

1,226 “mature” publications from the year 2023 related to AI in healthcare in the final analysis. Among these, the highest number of articles originated from the Imaging specialty (483), followed by Gastroenterology (86), and Ophthalmology (78). Analysis of data types revealed that image data was predominant, utilized in 75.2% of publications, followed by tabular data (12.9%) and text data (11.6%). Deep Learning models were extensively employed, constituting 59.8% of the models used.

For the LLM related publications,after exclusions, 584 publications were finally classified into the 26 different healthcare specialties and used for further analysis. The utilization of Large Language Models (LLMs), is highest in general healthcare specialties, at 20.1%, followed by surgery at 8.5%.

Related publication: Artificial Intelligence in Healthcare: 2023 Year in Review. Raghav Awasthi, Shreya Mishra, Piyush Mathur, et al. medRxiv 2024.02.28.24303482

A collection of 96,332 HAI documents (publications: 75,820, open research datasets: 638, patents: 11,226, grants: 6,113, and clinical trials: 2,535) during 2009 to 2021. On average, 75.12% of the documents were tagged with at least one label related to either health problems or AI technologies (with 92.9% of publications tagged).

Related publication: Xuanyu Shi,Daoxin Yin,Dongliang Cui,Jian Du, et al. A Bibliographic Dataset of Health Artificial Intelligence Research. Health Data Sci. 2024;4:0125.

A collection of seven newly-released datasets comprising both manually-curated and LLM-generated questions enriched for adversarial queries. Both  human assessment framework and dataset design process are grounded in an iterative participatory approach and review of possible biases in Med-PaLM 2 answers to adversarial queries.

Related publication: A Toolbox for Surfacing Health Equity Harms and Biases in Large Language Models. Stephen R. Pfohl, Heather Cole-Lewis, Ivor Horn, Karan Singhal ,et al. arXiv:2403.12025v1 [cs.CY] 

A curated database of FDA-cleared AI devices for medical image interpretation, a canonical task among the first to be clinically operationalized. 140 FDA clearances from January 2016 to October 2023 for 104 unique AI-enabled CAD products, with some products having multiple clearances over time.Epecifically focused on AI devices with use cases that are historically referred to as variations of “CAD”, a term that stems from computer-aided detection.

Related publication: McNamara, S.L., Yi, P.H. & Lotter, W. The clinician-AI interface: intended use and explainability in FDA-cleared AI devices for medical image interpretation. npj Digit. Med. 7, 80 (2024). https://doi.org/10.1038/s41746-024-01080-1

A dataset containing diverse set of respiratory sounds and rich meta-data, recorded between April-2020 and February-2022 from 2635 individuals (1819 SARS-CoV-2 negative, 674 positive, and 142 recovered subjects). The respiratory sounds contained nine sound categories associated with variants of breathing, cough and speech. The rich metadata contained demographic information associated with age, gender and geographic location, as well as the health information relating to the symptoms, pre-existing respiratory ailments, comorbidity and SARS-CoV-2 test status.

Related publication: Bhattacharya, D., Sharma, N.K., Dutta, D. et al. Coswara: A respiratory sounds and symptoms dataset for remote screening of SARS-CoV-2 infection. Sci Data10, 397 (2023). https://doi.org/10.1038/s41597-023-02266-0

Dermatology Images dataset of 1947 cases with annotated differential diagnoses(ddx) from multiple dermatologists across 419 conditions, associated risk categories for each condition, softmax prediction for 4 different models.

Related publications: 

[1] Stutz, D., et al. (2023).[Conformal prediction under ambiguous ground truth](https://openreview.net/forum?id=CAd6V2qXxc).TMLR.
[2] Stutz, D., et al. (2023).[Evaluating AI systems under uncertain ground truth: a case study in dermatology](https://arxiv.org/abs/2307.02191).ArXiv, abs/2307.02191.

The largest multi-organ dataset (by far) with the spleen, liver, kidneys, stomach, gallbladder, pancreas, aorta, and IVC annotated in 8,448 CT volumes, equating to 3.2 million slices.

Related publication: AbdomenAtlas-8K: Annotating 8,000 CT Volumes for Multi-Organ Segmentation in Three Weeks. NeurIPS 2023 • Chongyu Qu, Tiezheng Zhang, Hualin Qiao, Jie Liu, Yucheng Tang, Alan Yuille, Zongwei Zhou.arXiv:2305.09666 [eess.IV]

There are a total of 382 unique prompts, with 1146 total responses across three iterations of ChatGPT (GPT-3.5, GPT-4.0, GPT-4.0 with Internet). 19.8% of the responses were labeled as inappropriate, with GPT-3.5 accounting for the highest percentage at 25.7% while GPT-4.0 and GPT-4.0 with internet performing comparably at 16.2% and 17.5% respectively. 11.8% of responses were deemed appropriate with GPT-3.5 but inappropriate in updated models, highlighting the ongoing need to evaluate evolving LLMs.

Related publication: Chang CT, Farah H, Gui H, et al. Red Teaming Large Language Models in Medicine: Real-World Insights on Model Behavior. medRxiv. 2024:2024.2004.2005.24305411.

A fine-grained, expert-labeled dataset of 40 de-identified breast and pancreatic cancer progress notes at University of California, San Francisco, and assessed three recent LLMs (GPT-4, GPT-3.5-turbo, and FLAN-UL2) in zero-shot extraction of detailed oncological information from two narrative sections of clinical progress notes.

Related publication: Sushil, M., Kennedy, V., Mandair, D., Miao, B., Zack, T., & Butte, A. (2024). CORAL: expert-Curated medical Oncology Reports to Advance Language model inference (version 1.0). PhysioNet. https://doi.org/10.13026/v69y-xa45.

Dataset of 57 mock medical primary care consultations: audio, consultation notes, human utterance-level transcripts.

Related Publication:  Alex Papadopoulos Korfiatis, Francesco Moramarco, Radmila Sarac, and Aleksandar Savkov. 2022. PriMock57: A Dataset Of Primary Care Mock Consultations. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 588–598, Dublin, Ireland. Association for Computational Linguistics.

Llama2-MedTuned-Instructions is an instruction-based dataset developed for training language models in biomedical NLP tasks. It consists of approximately 200,000 samples, each tailored to guide models in performing specific tasks such as Named Entity Recognition (NER), Relation Extraction (RE), and Medical Natural Language Inference (NLI). This dataset represents a fusion of various existing data sources, reformatted to facilitate instruction-based learning.

Related publication: Exploring the Effectiveness of Instruction Tuning in Biomedical Language Processing. Omid Rohanian, Mohammadmahdi Nouriborji, David A. Clifton. arXiv:2401.00579 [cs.CL]

Related models: https://huggingface.co/nlpie/Llama2-MedTuned-7b ; https://huggingface.co/nlpie/Llama2-MedTuned-13b

The largest smartwatch-based dataset including Parkinson’s, other Movement Disorders and Healthy controls (n>400). over 5000 clinical assessment steps from 504 participants, including PD, DD, and healthy controls (HC).

Related publication: Varghese, J., Brenner, A., Fujarski, M. et al. Machine Learning in the Parkinson’s disease smartwatch (PADS) dataset. npj Parkinsons Dis. 10, 9 (2024).

The National Neighborhood Data Archive (NaNDA) is a publicly available data archive containing contextual measures for locations across the United States. NaNDA offers theoretically derived, spatially referenced, nationwide measures of the physical and social environment. This dataset is very useful for social determinants of health(SDOH) and public health research.

A multimodal case report dataset which contains data from 75,382 open access PubMed Central articles spanning the period from 1990 to 2023. The dataset includes 96,428 clinical cases, 135,596 images, and their corresponding labels and captions.

Related publication: Mauro Andrés Nievas Offidani, Claudio Augusto Delrieux,Dataset of clinical cases, images, image labels and captions from open access case reports from PubMed Central (1990–2023),Data in Brief,Volume 52,2024,110008,ISSN 2352-3409

Github: https://github.com/mauro-nievoff/MultiCaRe_Dataset/tree/main

INSPECT contains data from 19,438 patients, including CT images, sections of radiology reports, and structured electronic health record (EHR) data (including demographics, diagnoses, procedures, and vitals). Using our provided dataset, Stanford University, develop and release a benchmark for evaluating several baseline modeling approaches on a variety of important PE related tasks.INSPECT is the largest multimodal dataset for enabling reproducible research on strategies for integrating 3D medical imaging and EHR data.

Related publication: INSPECT: A Multimodal Dataset for Pulmonary Embolism Diagnosis and Prognosis. Shih-Cheng Huang, Zepeng Huo, Ethan Steinberg, Chia-Chun Chiang, Matthew P. Lungren, Curtis P. Langlotz, Serena Yeung, Nigam H. Shah, Jason A. Fries. arxiv.2311.10798[cs.LG]

FluSense platform collected and analyzed more than 350,000 waiting room thermal images and 21 million non-speech audio samples from the hospital waiting areas.

Related publication: Forsad Al Hossain, Andrew A. Lover, George A. Corey, Nicholas G. Reich, and Tauhidur Rahman. 2020. FluSense: A Contactless Syndromic Surveillance Platform for Influenza-Like Illness in Hospital Waiting Areas. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 4, 1, Article 1 (March 2020), https://doi.org/10.1145/3381014

MedAlign is a clinician-generated dataset for instruction following with electronic medical records.The MedAlign dataset contains:

• 1314 clinician-generated instructions, 983 after removing duplicates using ROUGE-L overlap;
• 276 longitudinal EHRs;
• 303 clinician-generated responses to instruction-EHR pairs.

Related publication: MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records. Scott L. Fleming, Alejandro Lozano,  Nigam H. Shah ,et al. https://arxiv.org/abs/2308.14089

EHRSHOT, which contains de-identified structured data from the electronic health records (EHRs) of 6,739 patients from Stanford Medicine. CLMBR-T-base, a 141M parameter clinical foundation model pretrained on the structured EHR data of 2.57M patients.15 few-shot clinical prediction tasks, enabling evaluation of foundation models on benefits such as sample efficiency and task adaptation.

Related publication: EHRSHOT: An EHR Benchmark for Few-Shot Evaluation of Foundation Models.Michael Wornow, Rahul Thapa, Ethan Steinberg, Jason A. Fries, Nigam H. Shah arXiv:2307.02028

The Scottish Medical Imaging Archive is a collection of population-based, routinely collected medical radiology images.This archive provides access to “analytics-ready” extracts for images between January 1, 2010, and August 31, 2018, which can be used for health care research and the development or validation of artificial intelligence algorithms.An archive of 57.3 million radiology studies linked to their medical records from the whole Scottish population.Modalities: Computerised Tomography (CT), Magnetic Resonance Imaging (MRI), Positron EmissionTomography (PET), Structured Reports (SRs).

Related publication: Nind, T., Sutherland, J., Krueger, S., Teviotdale, R., Gillen, K., Reel, P. S., Reel, S., Steele, D., Doney, A., Trucco, M., & Jefferson, E. (2023). The Scottish Medical Imaging Archive: 57.3 million Radiology Studies Linked to their Medical Records. Radiology: Artificial Intelligence. Advance online publication. https://doi.org/10.1148/ryai.220266

The COUGHVID dataset provides over 20,000 crowdsourced cough recordings representing a wide range of subject ages, genders, geographic locations, and COVID-19 statuses. Furthermore, experienced pulmonologists labeled more than 2,000 recordings to diagnose medical abnormalities present in the coughs, thereby contributing one of the largest expert-labeled cough datasets in existence that can be used for a plethora of cough audio classification tasks.

Related publication: Lara Orlandic, Tomas Teijeiro, & David Atienza. (2021). The COUGHVID crowdsourcing dataset: A corpus for the study of large-scale cough analysis algorithms (2.0) [Data set]. Zenodo. https://doi.org/10.5281/zenodo.4498364

MeQSum corpus of 1,000 summarized consumer health questions. In particular, authors show that semantic augmentation from question datasets improves the overall performance, and that pointer-generator networks outperform sequence-to-sequence attentional models on this task, with a ROUGE-1 score of 44.16%.

Related publication: Asma Ben Abacha and Dina Demner-Fushman. 2019. On the Summarization of Consumer Health Questions. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2228–2234, Florence, Italy. Association for Computational Linguistics.

This is a set of 100,000 non-overlapping image patches from hematoxylin & eosin (H&E) stained histological images of human colorectal cancer (CRC) and normal tissue.All images are 224×224 pixels (px) at 0.5 microns per pixel (MPP). Tissue classes are: Adipose (ADI), background (BACK), debris (DEB), lymphocytes (LYM), mucus (MUC), smooth muscle (MUS), normal colon mucosa (NORM), cancer-associated stroma (STR), colorectal adenocarcinoma epithelium (TUM).These images were manually extracted from N=86 H&E stained human cancer tissue slides from formalin-fixed paraffin-embedded (FFPE) samples from the NCT Biobank (National Center for Tumor Diseases, Heidelberg, Germany) and the UMM pathology archive (University Medical Center Mannheim, Mannheim, Germany). Tissue samples contained CRC primary tumor slides and tumor tissue from CRC liver metastases; normal tissue classes were augmented with non-tumorous regions from gastrectomy specimen to increase variability.

Dataset for estimation of muscle dysmorphia in individuals from Barranquilla, Colombia

Related publication: Isaac Kuzmar, José Consuegra, Jezael Jiménez, Einer López, José Hernández, Ana Noreña-Peña,Dataset for estimation of muscle Dysmorphia in individuals from Colombia,Data in Brief,Volume 31,2020,105967,ISSN 2352-3409,https://doi.org/10.1016/j.dib.2020.105967.

TDID (Thyroid Digital Image Database) is a freely accessible database of ultrasound images of thyroid nodules from National University of Columbia. Currently, this database has a group of B-mode ultrasound images, which include a complete annotation and diagnostic description of the suspicious images of thyroid lesions, made by expert radiologists. From March 2014 to date, information from 389 patients has been collected.

The Global Health Data Exchange (GHDx) is a catalog of global health and demographic data. The goal of the GHDx is to help people locate data by cataloging information about data including the topics covered, by providing links to data providers or explaining how to acquire the data, and in cases where we have permission, providing the data directly for download. Use the GHDx to research population census data, surveys, registries, indicators and estimates, administrative health data, and financial data related to health.

DATASUS provides information that can serve to support objective analyzes of the health situation, evidence-based decision making and the development of health action programs.Measuring the population’s health status is a tradition in public health. It began with the systematic recording of mortality and survival data (Vital Statistics – Mortality and Live Births). With advances in the control of infectious diseases (Epidemiological and Morbidity information) and with a better understanding of the concept of health and its population determinants, the analysis of the health situation began to incorporate other dimensions of the health status.

Data on morbidity, disability, access to services, quality of care, living conditions and environmental factors have become metrics used in the construction of Health Indicators, which translate into relevant information for the quantification and evaluation of health information.

This section also contains information on the population’s Health Care, registrations (Care Network),  hospital and outpatient networks, registration of health establishments, as well as information on financial resources and Demographic and Socioeconomic information.

All of Us Research Program collects data from a wide variety of sources, including surveys, electronic health records (EHRs), biosamples, physical measurements, and wearables like Fitbit.Most All of Us participants contribute biosamples such as blood and/or saliva. DNA from these samples is extracted and sent to genome centers for genomic analysis, including whole genome sequencing (WGS) and genome-wide genotyping.The All of Us Data and Research Center leverages the OMOP CDM to empower researchers by using existing, standardized vocabularies and a harmonized data representation. These factors enable connection to other ontologies, datasets, and tools that use the same codes or data model.

740,000+ participants, 400,000+ electronic health records, 520,000+ biosamples.

UK Biobank has collected and continues to collect extensive environmental, lifestyle, and genetic data on half a million participants.It includes data for:

• Imaging: Brain, heart and full body MR imaging, plus full body DEXA scan of the bones and joints and an ultrasound of the carotid arteries.
• Genetics: Whole genome sequencing for all 500,000 participants, whole exome sequencing for 470,000 participants, genotyping (800,000 genome-wide variants and imputation to 90 million variants).
• Health linkages: Linkage to a wide range of electronic health-related records, including death, cancer, hospital admissions and primary care records.
• Biomarkers: Data on more than 30 key biochemistry markers from all participants, taken from samples collected at recruitment and the first repeat assessment.
• Activity monitor: Physical activity data over a 7-day period collected via a wrist-worn activity monitor for 100,000 participants plus a seasonal follow-up on a subset.
• Online questionnaires: Data on a range of exposures and health outcomes that are difficult to assess via routine health records, including diet, food preferences, work history, pain, cognitive function, digestive health and mental health.
• Repeat baseline assessments: A full baseline assessment is undertaken during the imaging assessment of 100,000 participants.
• Samples: Blood & urine was collected from all participants, and saliva for 100,000.

The Respiratory Sound database was originally compiled to support the scientific challenge organized at Int. Conf. on Biomedical Health Informatics – ICBHI 2017.The Respiratory Sound Database contains audio samples, collected independently by two research teams in two different countries, over several years.The database consists of a total of 5.5 hours of recordings containing 6898 respiratory cycles, of which 1864 contain crackles, 886 contain wheezes, and 506 contain both crackles and wheezes, in 920 annotated audio samples from 126 subjects.

Related publication: Garcia-Mendez JP, Lal A, Herasevich S, Tekin A, Pinevich Y, Lipatov K, Wang H-Y, Qamar S, Ayala IN, Khapov I, et al. Machine Learning for Automated Classification of Abnormal Lung Sounds Obtained from Public Databases: A Systematic Review. Bioengineering. 2023; 10(10):1155. 

A manually constructed dataset where clinicians asked naturally occurring questions about radiology images and provided reference answers. Manual categorization of images and questions provides insight into clinically relevant tasks and the natural language to phrase them. The dataset contains 104 head axial single-slice CTs or MRIs, 107 chest x-rays, and 104 abdominal axial CTs. The final VQA-RAD dataset contains 3,515 total visual questions. Of these, 1,515 (43.1%) are free-form.

Related publication: Lau, J., Gayen, S., Ben Abacha, A. et al. A dataset of clinically generated visual questions and answers about radiology images. Sci Data5, 180251 (2018).

This dataset contains over 9,000 head CT scans, each labeled as normal or abnormal. Each scan contains a reconstructed image (stored in our institution’s PACS and saved as DICOMs) and a corresponding sinogram (simulated via GE’s CatSim software and saved as numpy arrays). The reconstructed images are 512×512 pixels with a variable number of axial slices per scan. The sinograms are 984×888 pixels with a variable number of axial slices per scan. The full dataset is 1.3T.

Related publication: Hooper SM, Dunnmon JA, Lungren MP, Mastrodicasa D, Rubin DL, Ré C, Wang A, Patel BN. Impact of Upstream Medical Image Processing on Downstream Performance of a Head CT Triage Neural Network. Radiol Artif Intell. 2021 Apr 28;3(4):e200229. doi: 10.1148/ryai.2021200229. PMID: 34350412

A diverse medical task dataset comprising 52,000 instruction response pairs and,MedInstruct-test, a set of clinician-crafted novel medical tasks,to facilitate the building and evaluation of future domain-specific instruction-following models.

Related publication:ALPACARE:INSTRUCTION-TUNED LARGE LANGUAGE MODELS FOR MEDICAL APPLICATION

MedShapeNet contains over 100,000 medical shapes, including bones, organs, vessels, muscles, etc., as well as surgical instruments.

Related publication: MedShapeNet – A Large-scale Dataset of 3D Medical Shapes for Computer Vision

OpenPath, a large dataset of 208,414 pathology images paired with natural language descriptions.

Related publication:  Huang, Z., Bianchi, F., Yuksekgonul, M. et al. A visual–language foundation model for pathology image analysis using medical Twitter. Nat Med (2023). https://doi.org/10.1038/s41591-023-02504-3

This is a dataset used in the Med-HALT research paper. Med-HALT provides a diverse multinational dataset derived from medical examinations across various countries and includes multiple innovative testing modalities. Med-HALT includes two categories of tests reasoning and memory-based hallucination tests, designed to assess LLMs’ problem-solving and information retrieval abilities. This research paper focuses on the challenges posed by hallucinations in large language models (LLMs), particularly in the context of the medical domain. The authors propose a new benchmark and dataset, Med-HALT (Medical Domain Hallucination Test), designed specifically to evaluate hallucinations.

EMBED contains 364,000 screening and diagnostic mammographic exams for 110,000 patients from four hospitals over an 8-year period. The EMBED AWS Open Data release represents 20% of the dataset divided into two equal cohorts at the patient level. This release of the dataset includes 2D and C-view images.

The MIMIC-IV-ECHO module contains more than 500,000 echocardiograms across 7,243 studies from 4,579 distinct patients.  This subset contains echocardiograms for patients who appear in the MIMIC-IV Clinical Database and were admitted between 2017 and 2019.

Related publication: Gow, B., Pollard, T., Greenbaum, N., Moody, B., Johnson, A., Herbst, E., Waks, J. W., Eslami, P., Chaudhari, A., Carbonati, T., Berkowitz, S., Mark, R., & Horng, S. (2023). MIMIC-IV-ECHO: Echocardiogram Matched Subset (version 0.1). PhysioNet. https://doi.org/10.13026/ef48-v217.

The Genome Aggregation Database (gnomAD), originally launched in 2014 as the Exome Aggregation Consortium (ExAC), is the result of a coalition of investigators willing to share aggregate exome and genome sequencing data from a variety of large-scale sequencing projects, and make summary data available for the wider scientific community.

• v4 release is composed of 730,947 exomes and 76,215 genomes (GRCh38)
• gnomAD v4 structural variants (SV) represent 63,046 genomes (GRCh38)
• gnomAD v4 copy number variants (CNV) represent variants in less than 1% of 464,297 exomes (GRCh38)

The CodiEsp corpus contains manually coded clinical cases. All documents are in Spanish language and CIE10 is the coding terminology (it is the Spanish version of ICD10-CM and ICD10-PCS). The CodiEsp corpus has been randomly sampled into three subsets: the train, the development, and the test set. The train set contains 500 clinical cases, and the development and test set 250 clinical cases each.

Related publication: Iker de la Iglesia, María Vivó, Paula Chocrón, Gabriel de Maeztu, Koldo Gojenola, Aitziber Atutxa, An open source corpus and automatic tool for section identification in Spanish health records,Journal of Biomedical Informatics,2023,104461,ISSN 1532-0464,https://doi.org/10.1016/j.jbi.2023.104461.

3 key datasets for endocscopy:

1. Kvasir-dataset-v2 contains 8,000 images, 8 classes, 1,000 images for upper and lower endoscopy in each class:

2.The Kvasir-Capsule dataset

3.Kvasir SEG dataset

Related publications:Konstantin Pogorelov, Kristin Ranheim Randel,Halvorsen, Kvasir, et al.: A Multi-Class Image Dataset for Computer Aided Gastrointestinal Disease Detection, In MMSys’17 Proceedings of the 8th ACM on Multimedia Systems Conference (MMSYS), Pages 164-169 Taipei, Taiwan, June 20-23, 2017.

It contains 21 videos with a total number of 5, 525 frames annotated and verified by medical doctors (experienced endoscopists). The videos are divided into four classes of predefined bowel-preparation qualities.

Related publication: Konstantin Pogorelov, Kristin Ranheim Randel, Thomas de Lange, Sigrun Losada Eskeland, Carsten Griwodz, Dag Johansen, Concetto Spampinato, Mario Taschwer, Mathias Lux, Peter Thelin Schmidt, Michael Riegler, PÃ¥l Halvorsen, Nerthus: A Bowel Preparation Quality Video Dataset, In MMSys’17 Proceedings of the 8th ACM on Multimedia Systems Conference, Pages 170-174, Taipei, Taiwan, June 20-23, 2017.

This dataset comprises 18,481 images extracted from 523 small bowel capsule endoscopy videos. It has annotated 12,3320 images with 23,033 disease lesions and combined with 6,161 normal mucosa images. The annotations are provided in YOLO format.

Related publication: Yokote A, Umeno J, Kawasaki K, Fujioka S, Fuyuno Y, Matsuno Y, Yoshida Y, Imazu N, Miyazono S, Moriyama T, Kitazono T, Torisu T. Small bowel capsule endoscopy examination and open access database with artificial intelligence: The SEE-artificial intelligence project. DEN Open. 2023 Jun 22;4(1):e258. doi: 10.1002/deo2.258. PMID: 37359150; PMCID: PMC10288072.

More than 80 open source healthcare datasets available through the AWS Open Data Sponsorship Program.

3 datasets:

• NHS UK Q/A, 24,665 question and answer pairs, Prompt used: f53cf99826, Generated via OpenGPT using data available on the NHS UK Website. Download here.(Click on view Raw data)
• NHS UK Conversations, 2,354 unique conversations, Prompt used: f4df95ec69, Generated via OpenGPT using data available on the NHS UK Website. Download here. (Click on view Raw data)
• Medical Task/Solution, 4,688 pairs generated via OpenGPT using GPT-4, prompt used: 5755564c19. Download here. (Click on view Raw data)

Data to predict the course of Parkinson’s disease (PD) using protein abundance data. The core of the dataset consists of protein abundance values derived from mass spectrometry readings of cerebrospinal fluid (CSF) samples gathered from several hundred patients. Each patient contributed several samples over the course of multiple years while they also took assessments of PD severity.This is a time-series code dataset with Kaggle’s time-series API.

The data series include three datasets, collected under distinct circumstances:

• The tDCS FOG (tdcsfog) dataset, comprising data series collected in the lab, as subjects completed a FOG-provoking protocol.
• The DeFOG (defog) dataset, comprising data series collected in the subject’s home, as subjects completed a FOG-provoking protocol
• The Daily Living (daily) dataset, comprising one week of continuous 24/7 recordings from sixty-five subjects. Forty-five subjects exhibit FOG symptoms and also have series in the defog dataset, while the other twenty subjects do not exhibit FOG symptoms and do not have series elsewhere in the data.

The National Institutes of Health’s All of Us Research Program is building one of the largest biomedical data resources of its kind.

600,000+ participants

350,000+ EHR records

450,000+ biomedical specimen data

3 metastatic cancer  datasets available through AWS API.

• Time Series Dataset – Each row in the time series dataset represents a point in time, in units of days indexed from each patient’s initial gamma knife radiosurgery. Dataset variables include clinical details related to medication changes, imaging timing/references to raw imaging files, procedure timing, clinical follow up, and outcomes.
• Individual Dataset – Each row represents an individual patient with demographic details and summary clinical data.
• Gamma Knife Details Dataset – Each row represents an individual gamma knife target to provide further details about available gamma knife radiosurgery.

The Dermofit Image Library is a collection of 1,300 focal high quality skin lesion images collected under standardised conditions with internal colour standards. The lesions span across ten different classes including melanomas, seborrhoeic keratosis and basal cell carcinomas. Each image has a gold standard diagnosis based on expert opinion (including dermatologists and dermatopathologists). Images consist of a snapshot of the lesion surrounded by some normal skin.The Dermofit Image Library is available under an academic licence. There is a one-off £75 licence fee associated with this product.

Related publication: Rees, Aldridge, Fisher, Ballerini (2013), A Color and Texture Based Hierarchical K-NN Approach to the Classification of Non-melanoma Skin Lesions, Color Medical Image Analysis, Lecture Notes in Computational Vision and Biomechanics 6 (M. E. Celebi, G. Schaefer (eds.))

A dataset of more than 100,000 chest X-ray scans that were retrospectively collected from two major hospitals in Vietnam. Out of this raw data,  18,000 images that were manually annotated by a total of 17 experienced radiologists with 22 local labels of rectangles surrounding abnormalities and 6 global labels of suspected diseases.

The EchoNet-Peds database includes 7,643 labeled echocardiogram videos and human expert annotations (measurements, tracings, and calculations) to provide a baseline to study cardiac motion and chamber sizes. The database includes patients ranging from 0-18 years (43% female) with a wide range of sizes.

Related publication: Reddy CD, Lopez L, Ouyang D, Zou JY, He B. Video-Based Deep Learning for Automated Assessment of Left Ventricular Ejection Fraction in Pediatric Patients. J Am Soc Echocardiogr. 2023 Feb 6:S0894-7317(23)00068-8. doi: 10.1016/j.echo.2023.01.015. Epub ahead of print. PMID: 36754100.

All BraTS multimodal scans are available as NIfTI files (.nii.gz) which were were acquired with different clinical protocols and various scanners from multiple (n=19) institutions.The overall survival (OS) data, defined in days, are included in a comma-separated value (.csv) file with correspondences to the pseudo-identifiers of the imaging data. The .csv file also includes the age of patients, as well as the resection status.

Related publication:B. H. Menze, A. Jakab, S. Bauer, J. Kalpathy-Cramer, K. Farahani, J. Kirby, et al. “The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS)”, IEEE Transactions on Medical Imaging 34(10), 1993-2024 (2015) DOI: 10.1109/TMI.2014.2377694

The data in this challenge contains whole-slide images (WSI) of hematoxylin and eosin (H&E) stained lymph node sections.Depending on the particular data set (see below), ground truth is provided:

• On a lesion-level: with detailed annotations of metastases in WSI.
• On a patient-level: with a pN-stage label per patient.

All ground truth annotations were carefully prepared under supervision of expert pathologists. WSI are provided as TIFF images. Lesion-level annotations are provided as XML files. For training, 100 patients will be provided and another 100 patients for testing.The test data set contains 500 slides. 1000 slides with 5 slides per patient .

The dataset contains 7,471 chest X-ray images in .png  file format and 3955 patients radiology text reports available in .XML format. Each image has been paired with four captions such as Impressions, Findings, Comparison and Indication that provide clear descriptions of the salient entities and events.

Original data source : https://openi.nlm.nih.gov/

Open-i provides access to over 3.7 million images from about 1.2 million PubMed Central^® articles; 7,470 chest x-rays with 3,955 radiology reports; 67,517 images from NLM History of Medicine collection; and 2,064 orthopedic illustrations.

A  synthetic dataset of brain images simulated across 42 different MR protocols and based on 500 different reference brains from the Human Connectome Project (HCP) (Van Essen et al., 2012), leading to 21,000 simulated brain images,

Related Publication: You S, Reyes M. Influence of contrast and texture based image modifications on the performance and attention shift of U-Net models for brain tissue segmentation. Frontiers in Neuroimaging. 2022;1.

An open-source data collection consisting a total of 955 T1-weighted MRIs (Magnetic Resonance Imaging) with manually segmented diverse lesions and metadata

Related publication: Liew, Sook-Lei. The Anatomical Tracings of Lesions after Stroke (ATLAS) Dataset – Release 2.0, 2021. Inter-university Consortium for Political and Social Research [distributor], 2022-08-08. https://doi.org/10.3886/ICPSR36684.v5

The dataset is a single-institutional, retrospective collection of 922 biopsy-confirmed invasive breast cancer patients, over a decade, having the following data components:

1. Demographic, clinical, pathology, treatment, outcomes, and genomic data: Collected from a variety of sources including clinical notes, radiology report, and pathology reports.
2. Pre-operative dynamic contrast enhanced (DCE)-MRI: Downloaded from PACS systems and de-identified for The Cancer Imaging Archive (TCIA) release in DICOM format.
3. Locations of lesions in DCE-MRI: Annotations on the DCE-MRI images by radiologists.
4. Imaging features from DCE-MRI: A set of 529 computer-extracted imaging features by inhouse software.

Related publication: Saha, A., Harowicz, M.R., Grimm, L.J., Kim, C.E., Ghate, S.V., Walsh, R. and Mazurowski, M.A., 2018. A machine learning approach to radiogenomics of breast cancer.

The National Health and Nutrition Examination Survey (NHANES) is a program of studies designed to assess the health and nutritional status of adults and children in the United States. The NHANES interview includes demographic, socioeconomic, dietary, and health-related questions. The survey examines a nationally representative sample of about 5,000 persons each year. Findings from this survey will be used to determine the prevalence of major diseases and risk factors for diseases.

This is an original dataset of stringency of public health policy measures that were adopted in response to COVID-19 worldwide by governments at national and sub-national levels. The data set covers governments’ policy responses between January 24, 2020 and December 31, 2020.

Related publication: Shvetsova, O., Zhirnov, A., Adeel, A.B. et al. Protective Policy Index (PPI) global dataset of origins and stringency of COVID 19 mitigation policies. Sci Data 9, 319 (2022). https://doi.org/10.1038/s41597-022-01437-9

Multiple datasets available:

1. silent-cchs-ecg: Diagnosing ‘silent’ heart attack (48,000 ECG waveforms)
2. brca-psj-path: Identifying high-risk breast cancer (175,000 biopsy slides)
3. arrest-ntuh-ecg: Subtyping cardiac arrest (24,106 ECG waveforms)
4. fracture-aimi-xray: Predicting fractures (64,000 chest x-rays)
5. covid-psj-xray: Emergency triage of Covid-19 patients (7,500 chest x-rays)

A dataset consisting of 53,449 audio samples (over 552 hours in total) crowd-sourced from 36,116 participants through our COVID-19 Sounds app. It also provides participants’ self-reported COVID-19 testing status with 2,106 samples tested positive.

Related publication: COVID-19 Sounds: A Large-Scale Audio Dataset for Digital Respiratory Screening

Related publication: Jain, S., Agrawal, A., Saporta, A., Truong, S. Q., Nguyen Duong, D., Bui, T., Chambon, P., Lungren, M., Ng, A., Langlotz, C., & Rajpurkar, P. (2021). RadGraph: Extracting Clinical Entities and Relations from Radiology Reports (version 1.0.0). PhysioNet. https://doi.org/10.13026/hm87-5p47.

200+ datasets of various types with links and papers.Includes search options for datatypes, language and more.

The PH² database includes the manual segmentation, the clinical diagnosis, and the identification of several dermoscopic structures, performed by expert dermatologists, in a set of 200 dermoscopic images.

Related publication: Mendonca T, Ferreira PM, Marques JS, Marcal AR, Rozeira J. PH² – a dermoscopic image database for research and benchmarking. Annu Int Conf IEEE Eng Med Biol Soc. 2013;2013:5437-40. doi: 10.1109/EMBC.2013.6610779. PMID: 24110966

Vision-based Fallen Person (VFP290K) dataset consists of 294,713 frames of fallen persons extracted from 178 videos, including 131 scenes in 49 locations. It demonstrated the effectiveness of the features through extensive experiments analyzing the performance shift based on object detection models.

Related publication: VFP290K: A Large-Scale Benchmark Dataset for Vision-based Fallen Person Detection

HiRID is a freely accessible critical care dataset containing data relating to almost 34 thousand adult patient admissions to the Department of Intensive Care Medicine of the Bern University Hospital, Switzerland (ICU), an interdisciplinary 60-bed unit admitting >6,500 patients per year. The dataset contains de-identified demographic information and a total of 681 routinely collected physiological variables, diagnostic test results and treatment parameters from almost 34 thousand admissions during the period from January 2008 to June 2016. Data is stored with a uniquely high time resolution of one entry every two minutes.

Related publication: Faltys, M., Zimmermann, M., Lyu, X., Hüser, M., Hyland, S., Rätsch, G., & Merz, T. (2021). HiRID, a high time-resolution ICU dataset (version 1.1.1). PhysioNet. https://doi.org/10.13026/nkwc-js72.

eICU Collaborative Research Database, a multi-center intensive care unit (ICU)database with high granularity data for over 200,000 admissions to ICUs monitored by eICU Programs across the United States.

Related publication: The eICU Collaborative Research Database, a freely available multi-center database for critical care research. Pollard TJ, Johnson AEW, Raffa JD, Celi LA, Mark RG and Badawi O. Scientific Data (2018). DOI: http://dx.doi.org/10.1038/sdata.2018.178.

The Medical Information Mart for Intensive Care (MIMIC)-IV database provided critical care data for over 40,000 patients admitted to intensive care units at the Beth Israel Deaconess Medical Center (BIDMC).

Related publication: Johnson, A., Bulgarelli, L., Pollard, T., Horng, S., Celi, L. A., & Mark, R. (2020). MIMIC-IV (version 0.4). PhysioNet. https://doi.org/10.13026/a3wn-hq05.

A dataset of paired Electroencephalography (EEG) and video-infrared eye tracking (ET) recordings from 356 subjects for more than 47 hours in total. A benchmark consisting of 3 evaluation tasks with increasing difficulty is introduced alongside the dataset.

Related publication: EEGEyeNet: a Simultaneous Electroencephalography and Eye-tracking Dataset and Benchmark for Eye Movement Prediction

Q-Pain, a dataset for assessing bias in medical QA in the context of pain management. 55 medical question-answer pairs across five different types of pain management: each question includes a detailed patient-specific medical scenario (“vignette”) designed to enable the substitution of multiple different racial and gender “profiles” and to evaluate whether bias is present when answering whether or not to prescribe medication.

Related publication: Logé, C., Ross, E., Dadey, D. Y. A., Jain, S., Saporta, A., Ng, A., & Rajpurkar, P. (2021). Q-Pain: A Question Answering Dataset to Measure Social Bias in Pain Management (version 1.0.0). PhysioNet. https://doi.org/10.13026/2tdv-hj07.

Dataset contributes significantly to the research community by providing 1) 1,256 combinations of relation annotations between 29 CXR anatomical locations (objects with bounding box coordinates) and their attributes, structured as a scene graph per image, 2) over 670,000 localized comparison relations (for improved, worsened, or no change) between the anatomical locations across sequential exams, as well as 3) a manually annotated gold standard scene graph dataset from 500 unique patients.

Related publication: Wu, J., Agu, N., Lourentzou, I., Sharma, A., Paguio, J., Yao, J. S., Dee, E. C., Mitchell, W., Kashyap, S., Giovannini, A., Celi, L. A., Syeda-Mahmood, T., & Moradi, M. (2021). Chest ImaGenome Dataset (version 1.0.0). PhysioNet. https://doi.org/10.13026/wv01-y230.

TDC includes 66 AI-ready datasets spread across 22 learning tasks and spanning the discovery and development of safe and effective medicines. TDC also provides an ecosystem of tools and community resources, including 33 data functions and diverse types of data splits, 23 strategies for systematic model evaluation, 17 molecule generation oracles, and 29 public leaderboards.

Related publication: Therapeutics Data Commons: Machine Learning Datasets and Tasks for Drug Discovery and Development

The RAD-ChestCT dataset is a imaging dataset developed by Duke MD/PhD student Rachel Draelos during her Computer Science PhD supervised by Lawrence Carin. The full dataset includes 35,747 chest CT scans from 19,661 adult patients. This Zenodo repository contains an initial release of 3,630 chest CT scans, approximately 10% of the dataset.

Related publication: Draelos et al., “Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes,” Medical Image Analysis 2021. DOI: 10.1016/j.media.2020.101857

A dataset consists of 70 melanoma and 100 naevus images from the digital image archive of the Department of Dermatology of the University Medical Center Groningen (UMCG) used for the development and testing of the MED-NODE system for skin cancer detection from macroscopic images. The file contains 170 images (70 melanoma and 100 nevi cases).

Related publications: I. Giotis, N. Molders, S. Land, M. Biehl, M.F. Jonkman and N. Petkov: “MED-NODE: A computer-assisted melanoma diagnosis system using non-dermoscopic images”, Expert Systems with Applications, 42 (2015), 6578-6585

BIGBIO a community library of 126+ biomedical NLP datasets currently covering 12 task categories and 10+ languages with • programmatic access. BIGBIO enables reproducible data-centric machine learning workflows, by focusing on programmatic access to datasets and their metadata in a uniform format.

Related Publication: BIGBIO: A Framework for Data-Centric Biomedical Natural Language Processing

A community-driven platform for reporting biomedical AI systems.

Related publication:  Matschinske, J., Alcaraz, N., Benis, A. et al. The AIMe registry for artificial intelligence in biomedical research. Nat Methods (2021), DOI: 10.1038/s41592-021-01241-0, https://rdcu.be/cv5H7

The dataset consists of 2,298 samples of six different types of skin lesions. Each sample consists of a clinical image and up to 22 clinical features including the patient’s age, skin lesion location, Fitzpatrick skin type, and skin lesion diameter. ll BCC, SCC, and MEL are biopsy-proven.In total, there are 1,373 patients, 1,641 skin lesions, and 2,298 images present in the dataset. The remaining ones may have clinical diagnosis according to a consensus of a group of dermatologists. In total, approximately 58% of the samples in this dataset are biopsy-proven.

Related publication: PAD-UFES-20: A skin lesion dataset composed of patient data and clinical images collected from smartphones

The database includes data from more than 705,000 patients, collected in more than 60 countries and 1,500 centres worldwide. Patient data are available from acute hospital admissions with COVID-19 and outpatient follow-ups. The data include signs and symptoms, pre-existing comorbidities, vital signs, chronic and acute treatments, complications, dates of hospitalization and discharge, mortality, viral strains, vaccination status, and other data.

Related publication: ISARIC-COVID-19 dataset: A Prospective, Standardized, Global Dataset of Patients Hospitalized with COVID-19

2201 images with diagnoses based on biopsy or clinical impression.174 disease classes for the model training.

Biomedical relation extraction dataset (BioRED) with multiple entity types (e.g. gene/protein, disease, chemical) and relation pairs (e.g. gene–disease; chemical–chemical) at the document level, on a set of 600 PubMed abstracts.

Related dataset: Ling Luo, et al. BioRED: a rich biomedical relation extraction dataset, Briefings in Bioinformatics, 2022

The RadImageNet database includes 1.35 million annotated CT, MRI, and ultrasound images of musculoskeletal, neurologic, oncologic, gastrointestinal, endocrine, and pulmonary pathology. The RadImageNet database contains medical images of 3 modalities, 11 anatomies, and 165 pathologic labels.

BRAX dataset provides 40,967 images, 24,959 imaging studies for 19,351 patients presenting to the Hospital Israelita Albert Einstein. All images have been verified by trained radiologists and de-identified to protect patient privacy. Fourteen labels were derived from free-text radiology reports written in Brazilian Portuguese using Natural Language Processing.

Related publication: BRAX, a Brazilian labeled chest X-ray dataset

The MONAI framework is the open-source foundation being created by Project MONAI. MONAI is a freely available, community-supported, PyTorch-based framework for deep learning in healthcare imaging.Project MONAI also includes MONAI Label, an intelligent open source image labeling and learning tool that helps researchers and clinicians collaborate, create annotated datasets, and build AI models in a standardized MONAI paradigm.

This collection comprises multi-parametric magnetic resonance imaging (mpMRI) scans for de novo Glioblastoma (GBM) patients from the University of Pennsylvania Health System, coupled with patient demographics, clinical outcome (e.g., overall survival, genomic information, tumor progression), as well as computer-aided and manually-corrected segmentation labels of multiple histologically distinct tumor sub-regions, computer-aided and manually-corrected segmentations of the whole brain, a rich panel of radiomic features along with their corresponding co-registered mpMRI volumes in NIfTI format.

630 patients, 3301 studies, 820,000 + images.

A platform for end-to-end development of machine learning solutions in biomedical imaging.Grand Challenge was developed in 2010 to make it easy for organizers of challenges to set up a website for a particular challenge and to bring all information on challenges in the domain of biomedical image analysis available at one place.This system has been operational since 2017 and has been used by over 300 challenges,70,000 users with more than 1000 algorithms.

A database for evaluating computerized image-based prediction of the 7-point skin lesion malignancy checklist. The dataset includes over 2000 clinical and dermoscopy color images, along with corresponding structured metadata tailored for training and evaluating computer aided diagnosis (CAD) systems.

Related publication: J. Kawahara, S. Daneshvar, G. Argenziano, and G. Hamarneh, “Seven-Point Checklist and Skin Lesion Classification using Multitask Multimodal Neural Nets,” IEEE Journal of Biomedical and Health Informatics, vol. 23, no. 2, pp. 538–546, 2019.

A free and open platform for validating and sharing BIDS-compliant MRI, PET, MEG, EEG, and iEEG data.720 public datasets and growing.

Webpage: https://openneuro.org/

The EchoNet-LVH dataset includes 12,000 labeled echocardiogram videos and human expert annotations (measurements, tracings, and calculations) to provide a baseline to study cardiac chamber size and wall thickness.

Related publication: High-Throughput Precision Phenotyping of Left Ventricular Hypertrophy with Cardiovascular Deep Learning

The database includes 154 conventional chest radiographs with a lung nodule (100 malignant and 54 benign nodules) and 93 radiographs without a nodule  The database also includes additional information such as; patient age, gender, diagnosis (malignant or benign), X and Y coordinates of nodule, simple diagram of nodule location. Lung nodule images were classified into five groups according to the degrees of subtlety.

Related publication:  Shiraishi J, Katsuragawa S, lkezoe J, et al: Development of a digital image database for chest radiographs with and without a lung nodule: Receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. AJR 174:71-74, 2000.

Data was collected from nine healthy volunteers during a study of propofol-induced unconsciousness.  For all subjects, approximately 3 hours of data were recorded while using target-controlled infusion protocol.Data includes continuous electrocardiogram (ECG); interventions included in the study for patient safety, such as administering phenylephrine (a vasopressor);heart rate variability (HRV) and electrodermal activity (EDA).

Related publication: Subramanian, S., Purdon, P., Barbieri, R., & Brown, E. (2021). Behavioral and autonomic dynamics during propofol-induced unconsciousness (version 1.0). PhysioNet. https://doi.org/10.13026/2rbc-1r03.

94 open access ophthalmological imaging datasets containing 507 724 images and 125 videos from 122 364 patients.

The PTB-XL ECG dataset is a large dataset of 21837 clinical 12-lead ECGs from 18885 patients of 10 second length. The raw waveform data was annotated by up to two cardiologists, who assigned potentially multiple ECG statements to each record. Total 71 different ECG statements conform to the SCP-ECG standard and cover diagnostic, form, and rhythm statements.

Related publication: Wagner, P., Strodthoff, N., Bousseljot, R.-D., Kreiseler, D., Lunze, F.I., Samek, W., Schaeffter, T. (2020), PTB-XL: A Large Publicly Available ECG Dataset. Scientific Data. https://doi.org/10.1038/s41597-020-0495-6

A collection of de-identified annotated medical imaging data to foster transparent and reproducible collaborative research. X-rays, CT scans, MRIs,Echocardiography and Dermatology images.

Diverse Dermatology Images (DDI) dataset—the first publicly available, deeply curated, and pathologically confirmed image dataset with diverse skin tones. The DDI was retrospectively selected from reviewing pathology reports in Stanford Clinics from 2010-2020. It has a total of 656 images representing 570 unique patients.

Datasets is a library for easily accessing and sharing datasets, and evaluation metrics for Natural Language Processing (NLP), computer vision, and audio tasks.Currently over 2658 datasets, and more than 34 metrics available.At least 13 datasets with “medical” term search.Load a dataset in a single line of code, and use our powerful data processing methods to quickly get your dataset ready for training in a deep learning model.

The DCSM dataset consists of 255 randomly selected and fully anonymized overnight lab-based PSG recordings from patients visiting the DCSM for the diagnosis of non-specific sleep related disorders. The DCSM dataset represents a diverse cohort of Danish patients with respect to demographic characteristics, diagnostic background and sleep/non-sleep related medication usage, collected between 2015-2018.

Two publicly-available datasets, DOD-H including 25 healthy volunteers and DOD-O including 55 patients suffering from obstructive sleep apnea (OSA). Both datasets have been scored by 5 sleep technologists from different sleep centers. We developed a framework to compare automated approaches to a consensus of multiple human scorers.

Related publication: A. Guillot, F. Sauvet, E. H. During and V. Thorey, “Dreem Open Datasets: Multi-Scored Sleep Datasets to Compare Human and Automated Sleep Staging,” in IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 28, no. 9, pp. 1955-1965, Sept. 2020, doi: 10.1109/TNSRE.2020.3011181.

375 heterogeneous cardiac magnetic resonance (CMR) datasets acquired by using four different scanner vendors in six hospitals and three different countries (Spain, Canada and Germany).

Related publication: V. M. Campello et al., “Multi-Centre, Multi-Vendor and Multi-Disease Cardiac Segmentation: The M&Ms Challenge,” in IEEE Transactions on Medical Imaging, vol. 40, no. 12, pp. 3543-3554, Dec. 2021, doi: 10.1109/TMI.2021.3090082.

TCIA data are organized as “collections”; typically these are patient cohorts related by a common disease (e.g. lung cancer), image modality or type (MRI, CT, digital histopathology, etc) or research focus. Supporting data related to the images such as patient outcomes, treatment details, genomics and image analyses are also provided when available. Over  100+ datasets, many of which are public.

Unstructured notes from the Research Patient Data Registry at Partners Healthcare,Boston,USA (originally developed during the i2b2 project). Clinical Natural Language Processing (NLP) data sets were originally created at a former NIH-funded National Center for Biomedical Computing (NCBC) known as i2b2: Informatics for Integrating Biology and the Bedside. Beginning in 2018, they are officially known as n2c2 (National NLP Clinical Challenges).

A publicly available EMR Question Answering (QA) corpus by creating a large-scale dataset, emrQA, using a novel semi-automated generation framework that allows for minimal expert involvement and re-purposes existing annotations available for other clinical NLP tasks.EmrQA has 1 million question-logical form and 400,000+ question answer evidence pairs. The dataset uses existing NLP task annotations from the i2b2 Challenge datasets.

Related publication: Pampari, A., Raghavan, P., Liang, J.J., & Peng, J. (2018). emrQA: A Large Corpus for Question Answering on Electronic Medical Records. EMNLP.

VSCapture, an open source tool developed in C# programming language on the .NET/Mono platform that allows the tool to run on Windows, Macintosh OS X, Linux Ubuntu operating systems.

Related Publication: Data acquisition from S/5 GE Datex anesthesia monitor using VSCapture.

Related Dataset: The University of Queensland Vital Signs Dataset.

The University of Queensland Vital Signs Dataset contains a wide range of patient monitoring data and vital signs that were recorded during 32 surgical cases where patients underwent anaesthesia at the Royal Adelaide Hospital.

12,625 whole-slide images (WSIs) of prostate biopsies were available for model development (the development set), 393 for performance evaluation during the competition phase (the tuning set), 545 as the internal validation set in the postcompetition phase and 1,071 for external validation from 6 different sites.

Related publication: Bulten, W., Kartasalo, K., Chen, PH.C. et al. Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge. Nat Med (2022). https://doi.org/10.1038/s41591-021-01620-2

This is a dataset from tertiary care medical college and hospital in India’s cardiology unit which had 14,845 admissions corresponding to 12,238 patients.

Related publication: Bollepalli, S.C.; Sahani, A.K.; Armoundas, A.A. ,et al. An Optimized Machine Learning Model Accurately Predicts In-Hospital Outcomes at Admission to a Cardiac Unit. Diagnostics 2022, 12, 241.

https://doi.org/10.3390/diagnostics12020241

The dataset included over 69,000 dermatology images.International Skin Imaging Collaboration (ISIC) is a global partnership that has organized the world’s largest repository of publicly available dermoscopic images, hosted the first public benchmarks for melanoma detection in dermoscopic images, titled “Skin Lesion Analysis Towards Melanoma Detection”, at the IEEE International Symposium of Biomedical Imaging (ISBI).

A dataset of 491 Head CT scans with 193,317 slices, anonymized dicoms for all the scans and the corresponding radiologists’ reads done by three radiologists with an experience of 8, 12 and 20 years in cranial CT interpretation respectively.

Related publication: Development and Validation of Deep Learning Algorithms for Detection of Critical Findings in Head CT scan.

Publicly available  suite of tailored deep neural network models for tackling different challenges ranging from screening to risk stratification to treatment planning for patients with the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). 

• Chest x-rays: 16,352 CXR images across 14,979 patients Click here
• Chest CT: 201,103 CT slices from 4,501 patients Click here
• Chest point-of-care ultrasound: 29,651 POCUS images Click here
• COVID-Net ICU:1925 records from 385 patients   Click here

Also,expanded to open source TB-Net initiative for tuberculosis screening, Fibrosis-Net initiative for pulmonary fibrosis progression prediction, and Cancer-Net initiative for cancer screening.

MIMIC-ED is a large, freely available database of emergency department (ED) admissions at the Beth Israel Deaconess Medical Center between 2011 and 2016. 448,972 ED stays with vital signs, triage information, medication reconciliation, medication administration, and discharge diagnoses available

Chest X-ray dataset with eye tracking and report dictation. Built on MIMIC Chest X-ray dataset.1,083 CXR images.

Related publication:

Karargyris, A., Kashyap, S., Lourentzou, I. et al. Creation and validation of a chest X-ray dataset with eye-tracking and report dictation for AI development.Sci Data 8, 92 (2021).

This database is the first multi-institutional, multi-national expert annotated COVID-19 imaging dataset.Annotated by three radiologists with majority vote adjudication by board certified radiologists,RICORD consists of 240 thoracic CT scans and 1,000 chest radiographs contributed from four international sites.

A comprehensive dataset of 6,388 surgical patients composed of intraoperative biosignals and clinical information from the Department of Anesthesiology and Pain Medicine, Seoul National University College of Medicine, Seoul, Korea .

The NuCLS dataset contains over 220,000 labeled nuclei from breast cancer images from The Cancer Genome Atlas( TCGA). These nuclei were annotated through the collaborative effort of pathologists, pathology residents, and medical students.

CheXpert is a  public dataset for chest radiograph interpretation, consisting of 224,316 chest radiographs of 65,240 patients from Stanford Hospital.

The GDC Portal is a platform from National Cancer Institute(NCI) with cancer related genomic data for 80,000+ cases.

Annotations by CARING’s (Centre for Advanced research in Imaging, Neuroscience and Genomics) expert radiologists on COVID-19 positive X-rays.The corresponding X-rays were released by Medical Imaging Data Bank of the Valencia region (BIMCV).

BIMCV-COVID19+ dataset is a large dataset with chest X-ray images  and computed tomography (CT) imaging of COVID-19 patients along with their radiographic findings, pathologies, polymerase chain reaction (PCR), immunoglobulin antibody tests and radiographic reports from Medical Imaging Databank in Valencian Region Medical Image Bank (BIMCV).These iterations of the database include 7377 CR, 9463 DX and 6687 CT studies.

Provided on Kaggle by the Vingroup Big Data Institute (VinBigData) aims to promote fundamental research and investigate novel and highly-applicable technologies.A dataset consisting of 18,000 images that have been annotated by experienced radiologists.

The EchoNet-Dynamic database includes 10,030 labeled echocardiogram videos and human expert annotations (measurements, tracings, and calculations) to provide a baseline to study cardiac motion and chamber sizes.

Related publication: Video-based AI for beat-to-beat assessment of cardiac function

941 sentences from 911 PubMed abstracts, annotated with PGx entities of interest (mainly genes variations, gene, drugs and phenotypes), and relationships between those.

40+ speciality classified list of open source datasets for healthcare with direct links to the datasets and more information.

More than a 100 healthcare related datasets from around the world, classified and annotated.

Dataset created for the purpose of continuing research into COVID-19. However with information from all 50 states and the District of Columbia, many US statistics can be compared.

The DILIrank dataset is an updated version of the LTKB Benchmark dataset. DILIrank consists of 1,036 FDA-approved drugs that are divided into four classes according to their potential for causing drug-induced liver injury (DILI).

Dataset for automatically segmenting and classifying corneal ulcers with 712 ocular staining images and the associated segmentation labels for flaky corneal ulcers.

4000+ healthcare datasets made available from Harvard University.Searchable and diverse.

Semi automatically generated nuclei instance segmentation and classification dataset with exhaustive nuclei labels across 19 different tissue types. The dataset consists of 481 visual fields, of which 312 are randomly sampled from more than 20K whole slide images at different magnifications, from multiple data sources.

A dataset with Images,mainly Chest X-rays from COVID-19 patients.

Multiple data sources for COVID-19 in a unified data model, ready for analysis at one place.

In response to the COVID-19 pandemic, the White House and a coalition of leading research groups have prepared the COVID-19 Open Research Dataset (CORD-19). CORD-19 is a resource of over 29,000 scholarly articles, including over 13,000 with full text, about COVID-19, SARS-CoV-2, and related coronaviruses.

This dataset has daily level information on the number of affected cases, deaths and recovery from 2019 novel coronavirus.

The data is available since 22 Jan, 2020.

Largest collection of Intracranial hemorrhage CT scans.874 035 images with expert annotations.

Reference: Construction of a Machine Learning Dataset through Collaboration: The RSNA 2019 Brain CT Hemorrhage Challenge

One of the most comprehensive source of many datasets in healthcare.Primarily from ICU patients.

https://physionet.org/about/database/

MIMIC – IV Dataset (https://physionet.org/content/mimiciv/0.4/)

Includes:

• Clinical datasets such as MIMIC,eICU collaborative and Pediatic ICU datasets.
• Waveform datasets with ECG,EEG,arterial blood pressure waveform.
• ECG datasets with various pathophysiologic changes and drug interactions.
• Fetal datasets including sounds and ECG.
• Gait and Balance datasets include gait dynamics for patients with various neurodegenerative disorders.
• Neuro and Myoelectic datasets with EEG,EMG and evoked potential waveforms.
• Image datasets with Chest X-rays and MRI images.
• Computed Tomography Images for Intracranial Hemorrhage Detection and Segmentation
• Miscellaneous datasets with text, language,posture and other datasets

Alzheimer’s disease patient’s imaging(MRI), clinical, genomic, and biomarker data for the purposes of scientific investigation, teaching, or planning clinical research studies.

http://adni.loni.usc.edu/data-samples/access-data/

RIM-ONE is a database for optic disc and cup segmentation evaluation by Medical Image Analysis group.

Contains data related to 23,376 intensive care unit and high dependency unit admissions at Amsterdam University Medical Center of adult patients from 2003-2016.

The FDA Adverse Event Reporting System (FAERS) is a database that contains adverse event reports, medication error reports and product quality complaints resulting in adverse events that were submitted to FDA

A repository of segmented cells from the thin blood smear slide images from the Malaria Screener research activity.The dataset contains a total of 27,558 cell images with equal instances of parasitized and uninfected cells.

1200 Retinal image dataset with annotation.

A de-identified dataset of retinal fundus images for glaucoma analysis (RIGA) derived from three sources with 750 original images and 4500 manual marked images

The public database contains 15 images of healthy patients, 15 images of patients with diabetic retinopathy and 15 images of glaucomatous patients.

39 images for development of vessel extraction algorithms suitable for retinal screening programmes.

Datasets from National Cancer Institute of over 54000 patients. They include data on participant characteristics, screening exam results, diagnostic procedures, lung cancer, and mortality. Images from over 75,000 CT screening exams are available. Over 1,200 pathology images from a subset of NLST lung cancer patients (~500 of over 2,000 patients) may be viewed.

Polysomnography dataset from NSRR for sleep studies.Large collection of deidentified physiologic signals perfect for ML development.

A large collection of multi-source dermatoscopic images of common pigmented skin lesions containing 10000 images.

Related publication:The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions

This open source repository has more than 400 datasets including healthcare(100+) and non-healthcare ones in searchable and categorized format.

CMS datasets provide US Medicare and Medicaid datasets.

ResDAC(The Research Data Assistance Center) provides free support to users of CMS datasets.Link:  https://www.resdac.org/learn

Center for Disease Control’s datasets.Useful for incidence,prevalance of various disorders and mortality data from across the US.

Agency for Healthcare Research and Quality’s HCUP datasets used to identify, track, and analyze US national trends in health care utilization, access, charges, quality, and outcomes.

UK government’s National Health services datasets.NHS choices datasets are useful for NLP and sentiment analysis both for GPs and hospitals.

Brain MRI datasets from Open Accesss series of Imaging Studies(OASIS).

Cancer epidemiology data available through NCI’s Surveillance,Epidemiology and End Result Program(SEER).

Cancer and genomics datasets.

A dataset of 14,000+ anonymized, radiologist labeled musculoskeletal X-rays from 12,000+ patients from Stanford ML group.

Related publication: https://arxiv.org/abs/1712.06957

1500+ knee MRI anonymized dataset from NYU.

One stop to learn Natural Language processing and more.

Related publication: Bird, Steven, Edward Loper and Ewan Klein (2009), Natural Language Processing with Python. O’Reilly Media Inc.

An excellent resource for trends and updates in AI, especially NLP by Elvis Saravia.

An excellent collection of articles on data science.

Google’s powerful search engine to assist with dataset search.

Over 100,000 anonymized chest x-ray images and their corresponding data from more than 30,000 patients, including many with advanced lung disease.

NIH release of  a dataset containing 32,000 CT scan images with annotated lesions  belonging to 4400 unique patients.

A CMS initiative to democratize research and development using beneficiary data.Greater than 70 million patient dataset available.

The link below is for NIH’s strategic plan for data science in healthcare.A must read for anyone using data in healthcare for research and innovation

Largest open source Chest X-Ray data set available through NIH’s clinical center.See the link in the article to access the data.Also available through GITHUB and KAGGLE.

One of the the largest and most advanced software development platform in the world with many datasets and repositories.

Kaggle is a great resource for de-identified datasets in healthcare.

A biomedical data search engine which searches for datasets across registries.

A place to store, share or find data.A platform for biomedical  research.

Detailed data repositories for biomedical research especially proteins and genetics.