NMT with Attention (Tensorflow): translate

[RyeTorres]

Hi,

Having trouble with this homework question.

• Process the sentence to translate and encode it
• Set the initial state of the decoder
• Get predictions of the next token (starting with the token) for a maximum of iterations (in case the token is never returned)
• Return the translated text (as a string), the logit of the last iteration (this helps measure how certain was that the sequence was translated in its totality) and the translation in token format.

Only the START and END comment tags

Cell 0:

# GRADED FUNCTION: translate
def translate(model, text, max_length=50, temperature=0.0):
    """Translate a given sentence from English to Portuguese

    Args:
      model (tf.keras.Model): The trained translator
      text (string): The sentence to translate
      max_length (int, optional): The maximum length of the translation. Defaults to 50.
      temperature (float, optional): The temperature that controls the randomness of the predicted tokens. Defaults to 0.0.

    Returns:
      tuple(str, np.float, tf.Tensor): The translation, logit that predicted <EOS> token and the tokenized translation
    """

    # Lists to save tokens and logits
    tokens, logits = [], []

    ### START CODE HERE ###
    # PROCESS THE SENTENCE TO TRANSLATE
    # Convert the original string into a tensor
    text = tf.strings.split([text])[0]  # Assuming text is a single string
    # Vectorize the text using the correct vectorizer
    context = text_vectorizer(text).to_tensor()  # Assuming a text_vectorizer function exists
    # Get the encoded context (pass the context through the encoder)
    context = model.encoder(context)

    # INITIAL STATE OF THE DECODER
    # First token should be SOS token with shape (1,1)
    next_token = tf.constant(sos_token, shape=(1, 1), dtype=tf.int32)  # Assuming sos_token is defined
    # Initial hidden and cell states should be tensors of zeros with shape (1, UNITS)
    state = [tf.zeros((1, units)), tf.zeros((1, units))]  # Assuming `units` is defined
    # You are done when you draw a EOS token as next token (initial state is False)
    done = False

    # Iterate for max_length iterations
    for _ in range(max_length):
        # Generate the next token
        next_token, logit, state, done = model.decoder(
            decoder=model.decoder,
            context=context,
            next_token=next_token,
            done=done,
            state=state,
            temperature=temperature
        )

        # If done then break out of the loop
        if done:
            break

        # Add next_token to the list of tokens
        tokens.append(next_token)
        # Add logit to the list of logits
        logits.append(logit)
    ### END CODE HERE ###

    # Concatenate all tokens into a tensor
    tokens = tf.concat(tokens, axis=-1)
    # Convert the translated tokens into text
    translation = tf.squeeze(tokens_to_text(tokens, id_to_word))  # Assuming tokens_to_text and id_to_word exist
    translation = translation.numpy().decode()

    return translation, logits[-1], tokens

Cell 1 (DO NOT EDIT/MODIFY):
#Running this cell multiple times should return the same output since temp is 0

temp = 0.0
original_sentence = "I love languages"

translation, logit, tokens = translate(trained_translator, original_sentence, temperature=temp)

print(f"Temperature: {temp}\n\nOriginal sentence: {original_sentence}\nTranslation: {translation}\nTranslation tokens:{tokens}\nLogit: {logit:.3f}")

---

'NameError Traceback (most recent call last)
Cell In[127], line 6
3 temp = 0.0
4 original_sentence = "I love languages"
----> 6 translation, logit, tokens = translate(trained_translator, original_sentence, temperature=temp)
8 print(f"Temperature: {temp}\n\nOriginal sentence: {original_sentence}\nTranslation: {translation}\nTranslation tokens:{tokens}\nLogit: {logit:.3f}")

Cell In[126], line 23, in translate(model, text, max_length, temperature)
21 text = tf.strings.split([text])[0] # Assuming text is a single string
22 # Vectorize the text using the correct vectorizer
---> 23 context = text_vectorizer(text).to_tensor() # Assuming a text_vectorizer function exists
24 # Get the encoded context (pass the context through the encoder)
25 context = model.encoder(context)

NameError: name 'text_vectorizer' is not defined'

[lars-unimelb]

I'm guessing what's happened here is that you forgot to import text_vectoriser from tensorflow, since it's not set. Maybe you forgot to copy something from the example?

[RyeTorres]

Hint: According to the instructor, the cell from the previous exercise provides a lot of insight.

# PROCESS SENTENCE TO TRANSLATE AND ENCODE

# A sentence you wish to translate
eng_sentence = "I love languages"

# Convert it to a tensor
texts = tf.convert_to_tensor(eng_sentence)[tf.newaxis]

# Vectorize it and pass it through the encoder
context = english_vectorizer(texts).to_tensor()
context = encoder(context)

# SET STATE OF THE DECODER

# Next token is Start-of-Sentence since you are starting fresh
next_token = tf.fill((1,1), sos_id)

# Hidden and Cell states of the LSTM can be mocked using uniform samples
state = [tf.random.uniform((1, UNITS)), tf.random.uniform((1, UNITS))]

# You are not done until next token is EOS token
done = False

# Generate next token
next_token, logit, state, done = generate_next_token(decoder, context, next_token, done, state, temperature=0.5)
print(f"Next token: {next_token}\nLogit: {logit:.4f}\nDone? {done}")