Important Token Indices in Notebook.ipynb

[sh-taheri]

Hello! Thanks for the great work! I'd appreciate if you can help me understand how you construct the important token indices as it seems confusing to me when looking at the notebook file.

Question 1.
Currnently, when I run the bellow code snippet:

for prompt in fg_prompts:
    ids = pipeline.tokenizer(prompt).input_ids
    indices = {
        j: tok
        for tok, j in zip(
            pipeline.tokenizer.convert_ids_to_tokens(ids), range(len(ids))
        )
    }
    print(indices)

I get the following result:

{0: '<|startoftext|>', 1: 'a</w>', 2: 'cat</w>', 3: 'and</w>', 4: 'a</w>', 5: 'dog</w>', 6: '<|endoftext|>'}
{0: '<|startoftext|>', 1: 'a</w>', 2: 'sks</w>', 3: 'dog</w>', 4: 'and</w>', 5: 'a</w>', 6: 'cat</w>', 7: '<|endoftext|>'}
{0: '<|startoftext|>', 1: 'a</w>', 2: 'sks</w>', 3: 'cat</w>', 4: 'and</w>', 5: 'a</w>', 6: 'dog</w>', 7: '<|endoftext|>'}

In the next code block, you've specified the important token indices:

important_token_indices = [
    [
        [2],    # Cat in Prompt 1
        [5],    # Cat in Prompt 2
        [2, 3]  # Cat in Prompt 3
    ],
    [
        [6],    # Dog in Prompt 1
        [2,3],  # Dog in Prompt 2
        [6]     # Dog in Prompt 3
    ],
]

But according to the above output:
index=5 in prompt 2 corresponds to token = 'a', and index=6 in prompt 1 corresponds to token='<|endoftext|>'.
Also, the ordering shouldn't be Dog first and then Cat?

Can you please elaborate on how you construct the indices?

Question 2.
In this example I assume the trigger words of the LoRA models are 'sks', in other words, according to the paper: L1 = 'sks', S1 = 'dog', L2='sks', and S2 = 'cat'. (Plz correct me if I am wrong).
If the LoRA names when tokenized contain more than one token, should we use their indices in this format? [.., n, n+1,..., n+k]

Thanks!

[ShunkaiZhou]

I have the same question

[vietvo89]

@sh-taheri and @ShunkaiZhou , could you run the notebook successfully? I got the same error as shown in the last cell of the notebook.