AI-generated medical data can sidestep usual ethics review, universities say - Nature

AI‑generated medical data and ethics review: context, controversies, and good practice

A recent Nature headline suggests some universities view research using AI‑generated (synthetic) medical data as outside the scope of “usual” ethics review. Here’s what that means, why it’s contentious, and how to govern it responsibly.

Note: I don’t have access to the specific Nature article referenced. The analysis below draws on widely used research‑ethics concepts, privacy law, and AI governance guidance. Details in any particular institution may differ.

Background: how “human‑subjects research” is defined

Institutional Review Boards (IRBs) or equivalent ethics committees typically review studies that involve living individuals from whom investigators obtain data or biospecimens through interaction or intervention, or obtain, use, study, analyze, or generate identifiable private information. In many jurisdictions:

  • Projects that do not involve interactions with people and do not use identifiable private information can be deemed “not human‑subjects research.”
  • Some projects can be “exempt” if they meet narrow criteria (for example, use of publicly available or properly de‑identified data), though many institutions still require an administrative determination.
  • The term “synthetic data” generally refers to data artificially generated by algorithms (e.g., GANs, diffusion models, LLMs) to mirror the statistical properties of real datasets without being one‑to‑one copies of real records.

Because synthetic datasets may not be “about” identifiable living individuals in a direct sense, some universities classify research using them as outside full IRB review. That can speed up work—but it raises real questions.

Why universities may treat AI‑generated medical data as exempt or “not human‑subjects research”

  • No direct interaction with patients: Researchers often work solely with algorithms and outputs, not with people.
  • Claimed absence of identifiability: If the synthetic data are not reasonably linkable to an individual, they may be viewed like de‑identified or anonymous data.
  • Regulatory precedents:
    • In the US, some IRBs rely on OHRP decision charts to determine “not human‑subjects research.”
    • HIPAA’s de‑identification standard guides when health data are no longer considered Protected Health Information for covered entities.
    • Under the EU’s GDPR, data that cannot be related to an identified or identifiable person (anonymised) fall outside the regulation’s scope.
  • Operational efficiency: Treating synthetic datasets as outside the usual review path can reduce administrative burden and accelerate exploratory or methods work.

Why this is controversial: risks and gray areas

  • Training‑data leakage and re‑identification: Poorly generated synthetic data can memorize and reproduce real records. Even if individual rows are not direct copies, rare‑condition combinations or quasi‑identifiers can enable linkage attacks, especially when combined with other datasets.
  • Membership inference: Adversaries may infer whether a particular person’s data were used to train a model, which can itself be sensitive.
  • Bias and representativeness: Synthetic data inherit and can amplify biases present in source data or generation pipelines, potentially misleading downstream research or models.
  • Consent and expectations: Patients who provided real data may not have consented to derivative synthetic generation, particularly for secondary uses or commercial sharing.
  • Boundary problems in workflows: Even if the primary analysis uses synthetic data, teams often fine‑tune models, validate results, or perform error analysis with small samples of real data—pulling the project back into human‑subjects territory.
  • De‑identification is not absolute: Technical claims of “anonymity” can outpace practical privacy risks. The bar is context‑dependent and changes as re‑identification techniques evolve.
  • Accountability gaps: Exemption from formal review can weaken documentation, risk assessment, and community oversight precisely when new failure modes are emerging.

Common scenarios and pain points

  • Synthetic EHR cohorts: Hospitals generate synthetic records for method development. Later, researchers evaluate model performance on a small set of real charts. The evaluation step likely requires IRB oversight.
  • Medical imaging augmentation: Labs use diffusion models to create chest X‑rays or CT scans for training. If the generator overfits, some images may contain patient‑unique artifacts.
  • Clinical text synthesis: LLMs produce synthetic clinical notes. If trained on non‑consented notes, outputs could leak sensitive patterns or rare diagnoses.
  • Data sharing to external partners: A university shares synthetic data with startups. Contract terms, technical safeguards, and disclosures to original data contributors become critical.

The evolving regulatory and policy landscape

  • United States:
    • Common Rule/OHRP: IRBs decide when a project is “not human‑subjects research” or exempt. Many institutions require a formal determination letter even for exempt work.
    • HIPAA: De‑identification (Safe Harbor or Expert Determination) governs PHI handling by covered entities; synthetic data derived from PHI still carry obligations if linkability risks remain.
    • NIST AI RMF: Encourages risk‑based governance, including documentation, adversarial testing, and monitoring of privacy, bias, and security risks.
  • European Union:
    • GDPR: Truly anonymised data fall outside GDPR; pseudonymised data do not. Supervisory authorities emphasize that anonymisation must be robust against “reasonable” re‑identification attempts.
    • EU AI Act: Introduces obligations for high‑risk AI systems, including health contexts, focusing on data governance, risk management, and transparency—relevant even when data are synthetic.
  • United Kingdom:
    • ICO anonymisation and synthetic data guidance: Stresses context‑specific risk analysis, testing for linkability, and governance controls.
    • HRA/IRB practice: “Is my study research?” tools and proportionate review mechanisms; many institutions allow administrative determinations for low‑risk or non‑research activities.

Good practice when using synthetic medical data

  • Seek an official determination: Even if you believe a project is exempt or “not human‑subjects research,” obtain a written determination from the IRB or ethics office.
  • Document provenance and pipeline: Record the real datasets used, the generation methods, parameter settings, privacy safeguards, and quality checks.
  • Assess privacy risks empirically:
    • Run membership‑inference and attribute‑inference tests.
    • Measure nearest‑neighbor distances to check for near‑duplicates of real records.
    • Evaluate re‑identification risk under plausible attacker models and auxiliary data.
  • Use privacy‑enhancing techniques: Consider differential privacy during training, controlled sampling, and post‑processing to reduce leakage.
  • Separate roles and environments: Keep synthetic‑data workspaces isolated. Control and log access to any real data used for training or validation.
  • Perform bias and utility checks: Quantify representativeness, subgroup performance, and calibration. Publish validation protocols so users understand limits.
  • Limit sensitive attributes and rare combinations: Apply k‑anonymity‑like constraints or censor extreme outliers that could re‑identify people or reveal stigmatizing traits.
  • Establish governance even if exempt: Create a lightweight ethics review or data governance checklist, including a Data Protection Impact Assessment where GDPR applies.
  • Clarify terms of use and sharing: License synthetic datasets with clear restrictions (e.g., no re‑identification attempts, security requirements, prohibition on linkage without approval).
  • Maintain provenance and watermarking where feasible: Tag synthetic records and models to signal origin and discourage misuse or confusion with real data.
  • Be transparent with stakeholders: Inform data contributors, oversight bodies, and the public about how synthetic data are created and used, especially for high‑impact or commercial applications.

A quick checklist for teams

  • IRB/ethics office consulted; written determination on file.
  • Legal basis for source data use is clear; consent terms reviewed.
  • Data generation method and parameters documented; version controlled.
  • Privacy testing: nearest‑neighbor checks, membership‑inference, linkage stress tests.
  • Bias, representativeness, and utility assessed; limits disclosed.
  • Security controls in place; access logged; synthetic and real data segregated.
  • Data sharing agreements and acceptable‑use terms defined.
  • Plan for monitoring, incident response, and dataset/model retirement.

Potential benefits—when done right

  • Access and scale: Enables broader collaboration and method development without exposing raw patient records.
  • Reproducibility: Facilitates open benchmarking and education in settings where real data cannot be shared.
  • Privacy by design: Properly engineered synthetic data can lower privacy risks relative to sharing de‑identified real data.
  • Support for rare conditions: Augments datasets where cases are scarce, reducing overfitting and improving model learning—if carefully validated.

Key questions for stakeholders

  • For investigators: Could any project phase (validation, error analysis, deployment) reintroduce identifiable data? Have we measured leakage and bias?
  • For IRBs/ethics committees: Do our policies distinguish between de‑identification and synthesis? Do we offer clear pathways for exemption determinations plus governance expectations?
  • For data protection officers/legal counsel: Does synthetic generation constitute further processing? Are DPIAs or transfer impact assessments required?
  • For sponsors/publishers: Are datasets and methods sufficiently documented for reproducibility and risk assessment?

Conclusion

Classifying research on AI‑generated medical data as outside “usual” ethics review can be defensible under current rules—but it is not a free pass. Synthetic data exist on a spectrum of privacy, utility, and risk. Responsible programs treat exemption as a trigger for proportionate governance: obtain formal determinations, document pipelines, test for leakage and bias, constrain sharing, and engage oversight. That approach preserves the benefits of synthetic data—faster science, broader access—while honoring the ethical commitments that make medical research worthy of public trust.

Further reading (general resources)

  • US OHRP Decision Charts: https://www.hhs.gov/ohrp/regulations-and-policy/decision-charts/index.html
  • HIPAA De‑identification Guidance: https://www.hhs.gov/hipaa/for-professionals/privacy/special-topics/de-identification/index.html
  • NIST AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
  • UK ICO Anonymisation and PETs guidance: https://ico.org.uk/for-organisations/uk-gdpr-guidance-and-resources/anonymisation/
  • UK HRA “Is my study research?” tool: https://www.hra-decisiontools.org.uk/research/
  • Article 29 WP/EDPB guidance on anonymisation (overview): https://edpb.europa.eu/our-work-tools/general-guidance