Meaning of the attributions obtained from the selfies encoder

[IlPakoZ]

Hello! I am working on a research project using your library and I have some questions. I have written this minimal code:

import selfies as sf

smiles = "Oc1cc(O)c2c(c1)O[C@H](c1ccc(O)c(O)c1)[C@H](O)C2"
selfies, attribution = sf.encoder(smiles, attribute=True)
print("SELFIES:", selfies, "\n")

for attrib in attribution:
    print(attrib)

The output of my code is the following:

SELFIES: [O][C][=C][C][Branch1][C][O][=C][C][=Branch1][Ring2][=C][Ring1][#Branch1][O][C@H1][Branch1][#C][C][=C][C][=C][Branch1][C][O][C][Branch1][C][O][=C][Ring1][Branch2][C@H1][Branch1][C][O][C][Ring1][S]

AttributionMap(index=0, token='[O]', attribution=[Attribution(index=0, token='O')])
AttributionMap(index=1, token='[C]', attribution=[Attribution(index=1, token='c')])
AttributionMap(index=2, token='[=C]', attribution=[Attribution(index=3, token='c')])
AttributionMap(index=3, token='[C]', attribution=[Attribution(index=4, token='c')])
AttributionMap(index=6, token='[O]', attribution=[Attribution(index=6, token='O')])
AttributionMap(index=5, token='[C]', attribution=[Attribution(index=6, token='O')])
AttributionMap(index=5, token='[Branch1]', attribution=[Attribution(index=6, token='O')])
AttributionMap(index=7, token='[=C]', attribution=[Attribution(index=8, token='c')])
AttributionMap(index=8, token='[C]', attribution=[Attribution(index=10, token='c')])
AttributionMap(index=11, token='[=C]', attribution=[Attribution(index=12, token='c')])
AttributionMap(index=12, token='[Ring1]', attribution=None)
AttributionMap(index=13, token='[#Branch1]', attribution=None)
AttributionMap(index=10, token='[Ring2]', attribution=[Attribution(index=12, token='c')])
AttributionMap(index=10, token='[=Branch1]', attribution=[Attribution(index=12, token='c')])
AttributionMap(index=14, token='[O]', attribution=[Attribution(index=15, token='O')])
AttributionMap(index=15, token='[C@H1]', attribution=[Attribution(index=16, token='[C@H]')])
AttributionMap(index=18, token='[C]', attribution=[Attribution(index=18, token='c')])
AttributionMap(index=19, token='[=C]', attribution=[Attribution(index=20, token='c')])
AttributionMap(index=20, token='[C]', attribution=[Attribution(index=21, token='c')])
AttributionMap(index=21, token='[=C]', attribution=[Attribution(index=22, token='c')])
AttributionMap(index=8, token='[O]', attribution=[Attribution(index=24, token='O')])
AttributionMap(index=23, token='[C]', attribution=[Attribution(index=24, token='O')])
AttributionMap(index=23, token='[Branch1]', attribution=[Attribution(index=24, token='O')])
AttributionMap(index=25, token='[C]', attribution=[Attribution(index=26, token='c')])
AttributionMap(index=12, token='[O]', attribution=[Attribution(index=28, token='O')])
AttributionMap(index=27, token='[C]', attribution=[Attribution(index=28, token='O')])
AttributionMap(index=27, token='[Branch1]', attribution=[Attribution(index=28, token='O')])
AttributionMap(index=29, token='[=C]', attribution=[Attribution(index=30, token='c')])
AttributionMap(index=30, token='[Ring1]', attribution=None)
AttributionMap(index=31, token='[Branch2]', attribution=None)
AttributionMap(index=17, token='[#C]', attribution=[Attribution(index=18, token='c')])
AttributionMap(index=17, token='[Branch1]', attribution=[Attribution(index=18, token='c')])
AttributionMap(index=32, token='[C@H1]', attribution=[Attribution(index=33, token='[C@H]')])
AttributionMap(index=35, token='[O]', attribution=[Attribution(index=35, token='O')])
AttributionMap(index=34, token='[C]', attribution=[Attribution(index=35, token='O')])
AttributionMap(index=34, token='[Branch1]', attribution=[Attribution(index=35, token='O')])
AttributionMap(index=36, token='[C]', attribution=[Attribution(index=37, token='C')])
AttributionMap(index=37, token='[Ring1]', attribution=None)
AttributionMap(index=38, token='[S]', attribution=None)

I have some perplexities about those results. I am trying to obtain a mapping that goes from each SELFIES atom to corresponding SMILES atom (only atoms, not special tokens), and I have a necessity to do it only through the selfies.encoder, not decoder. I need the original SMILES to stay exactly the same, and the decoder doesn't guarantee the original SMILES to be reconstructed from the SELFIES, as you mentioned in section 4.1 of your code paper.

When I use the encoder, I notice that indices from "0" to "3" correspond to the SELFIES tokens. The 4th index is missing but there are two "5" indices.

AttributionMap(index=5, token='[C]', attribution=[Attribution(index=6, token='O')])
AttributionMap(index=5, token='[Branch1]', attribution=[Attribution(index=6, token='O')])

I assume the code is implemented in such a way that only for BranchN tokens, BranchN and the next N token share the same indices. I can manually offset those cases for my purpose so it's not a big problem. However, there are some cases that I do not fully understand. For example:

# Ring and atom share the same index
AttributionMap(index=12, token='[Ring1]', attribution=None)
AttributionMap(index=12, token='[O]', attribution=[Attribution(index=28, token='O')])

# This ring has unique index instead
AttributionMap(index=30, token='[Ring1]', attribution=None)

Why does the first [Ring1] share index with the token [O]? Has it something to do with how the ring closes? But if so, why doesn't the other ring I mentioned not share index with anything?

Another doubt arises here. In the output you also find:

AttributionMap(index=8, token='[C]', attribution=[Attribution(index=10, token='c')])
AttributionMap(index=8, token='[O]', attribution=[Attribution(index=24, token='O')])

Those atoms are not a part of any branch. However, they have the same index, but correspond to different tokens in the SELFIES, according to property "token". In the original SELFIES, the token with index 8 is the first one, [C]. Why does this happen?

Maybe the indices don't correspond to the tokens because they are not supposed to correspond, but if then, is there a way to get those mappings while preserving the original SMILES? Are you aware of an easier way to do this?

I am currently using the latest version (2.2.0) of the library. Thank you in advance and happy holidays!