Font Size: 

This paper presents experimentation results on the effect of encoder-decoder stack depth to performance of vanilla transformer model used as a neural machine translation of low-resource languages. In this study, a pretrained transformer model is fine-tuned using a parallel corpus of Indonesian and Sundanese languages. The experiment results showed that performances of vanilla transformer model with 2, 4, or 6 stack depth are higher than performance of the model with 8 stack depth. In particular, the highest performances achieved by the transformer model with 2 stack depth are: 0.99 training accuracy, 0.97 validation accuracy, and 0.99 testing similarity.

transformer model, machine translation, deep learning