Tutorial: Transformer on Google Colab for Machine Translation (2)

A Transformer Tutorial on Google Colab, Provided by Google TensorFlow

Transformer Tutorial on Colab, Provided by Google TensorFlow

• Recently, I tried another Transformer example on Google Colab, provided by Google TensorFlow.
• Initially, I found a version that there is a Python library issue needs to be addressed. (so called v2, Here)
• Later on, I found the one that without any errors. (so called v1, Here)

1. Running v1 (No Error)
2. Python Library Issue When Running v2

1. Running v1 (No Error)

• The path here is:

Google Colaboratory

• The Transformer here got 4 layers for encoder and decoder.

1.1. Training

Training the Transformer

• About 70–80 seconds are used to train for each epoch. 20 epochs are used in the tutorial, and it can be modified.

1.2. Inference

Portuguese-to-English Machine Translation (The blanket is blanked just because the notebook is disconnected)

1.3. Inference with Attention Visualization

Portuguese-to-English Machine Translation with Attention Visualization

1.4. Model Save & Load

Model Save & Load (The blanket is blanked just because the notebook is disconnected)

• It also provides codes to save and load model.

2. Python Library Issue When Running v2

• The path here is:

Google Colaboratory

• The Transformer here also got 4 layers for encoder and decoder, though it describes that it uses 6 layers.

2.1. Error & Code Modification

Error Occurs When Running the Tokenizer

• Error occurs due to the deprecation of “tfds.features.text”.
• But we can run it by modifying “tfds.features.text” to “tfds.deprecated.text”. (It maybe changed again in the future.)
• Run the notebook again:

Successfully Running the Tokenizer

2.2. Training

Training the Transformer

• About 90 seconds are used to train for each epoch. 20 epochs are used in the tutorial, and it can be modified.

2.3. Inference

Portuguese-to-English Machine Translation

• Portuguese-to-English machine translation is performed after training.

2.4. Inference with Attention Visualization

Portuguese-to-English Machine Translation With Visualization

• There are also attention visualization provided.

• Just directly running the one without errors (v1) is okay though I tried 2 versions.
• (The above information is subject to change due to updates in the future.)

References

[v1] Transformer Colab from TensorFlow (No error one.)
[v2] Transformer Colab from TensorFlow (One error that can be fixed.)

Models

[Transformer from D2L.ai] [Transformer from Google TensorFlow]

My Other Previous Paper Readings & Tutorials