Skip to content

Conversation

@ghost
Copy link

@ghost ghost commented Oct 9, 2025

Description

This pull request updates the drift detection code to improve compatibility with Python 3.12+. Specifically:

  • Ensures that imports for scipy.stats functions (chi2, ks_2samp, wasserstein_distance) and KstestResult are up to date.
  • Adjusts _calculate_ks function type hints to use KstestResult.
  • Validates datasets and columns more robustly, including checks for empty series and identical datasets.
  • Cleans up redundant imports and prepares the code for Python 3.13+.

These changes do not alter the core drift calculation logic, which includes:

  • Categorical drift: PSI and Chi-Squared tests
  • Numerical drift: KS test and Earth Mover’s Distance
  • Prediction drift: PSI, Chi-Squared, KS, and EMD for classification and regression models

The code has not yet been fully tested on Python 3.13, so further testing is required to confirm full compatibility.

Related Issue

N/A

Type of Change

  • 📚 Examples / docs / tutorials / dependencies update
  • 🔧 Bug fix (non-breaking change which fixes an issue)
  • 🥂 Improvement (preparation for future Python compatibility)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to change)
  • 🔐 Security fix

Checklist

  • I've read the CODE_OF_CONDUCT.md document.
  • I've read the CONTRIBUTING.md guide.
  • I've written tests for all new methods and classes that I created. (Not applicable, only imports updated / compatibility improvements)
  • I've updated the docstrings in Google format where needed. (No functional changes made)
  • I've updated the pdm.lock by running pdm update-lock if dependencies changed. (Not applicable)

@kevinmessiaen kevinmessiaen self-requested a review October 10, 2025 05:32
Copy link
Member

@kevinmessiaen kevinmessiaen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your fix @kaiprvn

This look good, let's just run pre-commit hook to ensure proper code formatting

@ghost ghost closed this Oct 11, 2025
@ghost ghost deleted the fix-deprecated-scipy branch October 11, 2025 17:45
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant