Skip to content

Error when requesting a very large number of responses #3

@ravimody

Description

@ravimody

get_similar('user', user, 'product', 100000000)
returns

all2vec/__init__.py in get_similar(self, entity_type, entity_id, match_type, num_similar, oversample, normalize)
    168         match_vector = self.get_vector(entity_type, entity_id)
    169         return self.get_similar_vector(
--> 170             match_vector, match_type, num_similar, oversample, normalize)
    171
    172     def get_scores_vector(self, vector, match_type, match_id_array, normalize):

all2vec/__init__.py in get_similar_vector(self, match_vector, match_type, num_similar, oversample, normalize)
    156
    157         similar_items = self._annoy_objects[match_type].get_nns_by_vector(
--> 158             match_vector, num_similar, search_k)
    159         # compute inner products, and sort
    160         scores = self.get_scores_vector(

all2vec/__init__.py in get_nns_by_vector(self, vec, n, search_k)
     67         """Get nearest neighbors from an input vector."""
     68         nns = self._ann_obj.get_nns_by_vector(vec, n, search_k)
---> 69         return [self._ann_map_inv[x] for x in nns]
     70
     71     def get_item_vector(self, entity_id):

TypeError: 'NoneType' object is not iterable

Digging deeper, it looks like all2vec is not properly capturing this ANNOY error

> u.get_nns_by_vector([1] * num_dimensions, 100000000, search_k = 100000000 * 100)
---------------------------------------------------------------------------
OverflowError                             Traceback (most recent call last)
OverflowError: signed integer is greater than maximum

This error will come up whenever n or search_k is larger than the maximum integer.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions