BUG: MultiIndex.isin raising TypeError on iterator elements - #66540
Open
guptaishaan wants to merge 1 commit into
Open
BUG: MultiIndex.isin raising TypeError on iterator elements#66540guptaishaan wants to merge 1 commit into
guptaishaan wants to merge 1 commit into
Conversation
The validation loop added for GH#20252 / GH#26622 calls len(value) on anything is_list_like accepts, but that includes iterators, which have no __len__. Elements that are iterators used to reach MultiIndex.from_tuples, which handled them; now they raise "object of type 'list_iterator' has no len()" first. Materialize iterator elements with tuple() before the loop, so they can be length-checked and are still intact when from_tuples consumes them. Side effect: values is now always a list at that point, so an ndarray argument goes through to_object_array_tuples rather than tuples_to_object_array. 2-D ndarray arguments, which previously raised a buffer error, now behave like the equivalent list of rows. No input that previously returned a result changes. Closes pandas-dev#66514
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #66514
The
MultiIndex.isinvalidation loop added for GH#20252 / GH#26622 callslen(value)on anything
is_list_likeaccepts, but that includes iterators, which have no__len__. Elements that are iterators used to reachMultiIndex.from_tuples, whichhandled them; now they raise
TypeError: object of type 'list_iterator' has no len()first.
Fix: materialize iterator elements with
tuple(...)before the loop, so they can belength-checked and are still intact when
from_tuplesconsumes them.is_iteratorwas already imported in the module.
Verified on Linux, CPython 3.13, numpy 2.4, built from this branch with meson-python:
3.1.0.dev0+1420.g7986b42596),whose
multi.pyis byte-identical to current main.mi.isin({iter([3])})now returns[False False True], the same as released 3.0.3.TypeErrorand pass with the patch.pandas/tests/indexes/(16517 passed), every-k isintest in the repo (471passed), and the
multi.pydoctests all pass. ruff check and ruff format clean.One side effect worth flagging. Replacing
valueswith a list changes which branch offrom_tupleshandles an ndarray argument, somi.isin(np.array([[1, "a"], [3, "c"]]))now behaves like the equivalent list of rows instead of raising
ValueError: Buffer has wrong number of dimensions. Only inputs that previously raised are affected, nothingthat returned a result changes. Happy to add an
if any(is_iterator(v) for v in values):guard if you would rather keep that error path.No whatsnew entry: the regression is unreleased, it arrived with the 3.1.0 entry for
GH#20252 / GH#26622.
Not verified: full test suite, mypy/pyright, non-ruff pre-commit hooks.
Thanks to @loicdiridollou for the report and the minimal reproducer.