Tutorial: Transformer on Google Colab for Machine Translation (2)

A Transformer Tutorial on Google Colab, Provided by Google TensorFlow

Transformer Tutorial on Colab, Provided by Google TensorFlow
  • Recently, I tried another Transformer example on Google Colab, provided by Google TensorFlow.
  • Initially, I found a version that there is a Python library issue needs to be addressed. (so called v2, Here)
  • Later on, I found the one that without any errors. (so called v1, Here)
  1. Running v1 (No Error)
  2. Python Library Issue When Running v2

1. Running v1 (No Error)

  • The path here is:
  • The Transformer here got 4 layers for encoder and decoder.

1.1. Training

Training the Transformer
  • About 70–80 seconds are used to train for each epoch. 20 epochs are used in the tutorial, and it can be modified.

1.2. Inference

Portuguese-to-English Machine Translation (The blanket is blanked just because the notebook is disconnected)

1.3. Inference with Attention Visualization

Portuguese-to-English Machine Translation with Attention Visualization

1.4. Model Save & Load

Model Save & Load (The blanket is blanked just because the notebook is disconnected)
  • It also provides codes to save and load model.

2. Python Library Issue When Running v2

  • The path here is:
  • The Transformer here also got 4 layers for encoder and decoder, though it describes that it uses 6 layers.

2.1. Error & Code Modification

Error Occurs When Running the Tokenizer
  • Error occurs due to the deprecation of “tfds.features.text”.
  • But we can run it by modifying “tfds.features.text” to “tfds.deprecated.text”. (It maybe changed again in the future.)
  • Run the notebook again:
Successfully Running the Tokenizer

2.2. Training

Training the Transformer
  • About 90 seconds are used to train for each epoch. 20 epochs are used in the tutorial, and it can be modified.

2.3. Inference

Portuguese-to-English Machine Translation
  • Portuguese-to-English machine translation is performed after training.

2.4. Inference with Attention Visualization

Portuguese-to-English Machine Translation With Visualization
  • There are also attention visualization provided.
  • Just directly running the one without errors (v1) is okay though I tried 2 versions.
  • (The above information is subject to change due to updates in the future.)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store