Q: The static nature of the context vector in the encoder-decoder model is the bottleneck since it must represent every thing about the meaning of the source text, attention mechanism can be the solution (page 204-206), Is this attention mechanism language dependent (affected by the nature of the langauge) or not?[DH]
A: As far as I know, there are no language dependent solutions. However, it is based on the tokens one has chosen as input/output items. In this sense it depends on the kind of tokens which deemed most appropriate for the language considered.
Thinking about possible modifications of the attention mechanisms according to the languages involved seems to me to be a quite interesting research question.
    
Q: in Amharic to English translation "I called her" can be translated in one word (ጠራኋት). how encoder-decoder understands subject verb agreement and merge this in to one word.[YM]
A: The encoder computes a distributed representation for the whole input sentence. If this representation is good enough it should capture the information of all input words. If this is the case, the decoder should be able to transform it into a corresponding Amharic sentence, irrespective of how many words are needed for that purpose. The decoder terminates if al aspects encoded in the context vector have been properly verbalized. 

Q: When Encoder converts the input into fixed length vector, does it mean it converts all inputs in the same length? example input1-> "hello",  input2-> "how are you". does it mean input1 and input2 will have the same length? [YM]
A: Yes, this is the case. You have to distinguish the length of the sentence which is unlimited and the number of dimensions of a vector representation, which is fixed as determined by the network developer. It is exactly the task of the encoder to (e.g. recurrently) build up a fixed-length vector representation for the variable-length input sentence.


-- WolfgangMenzel - 09 Mar 2023
 
This site is powered by FoswikiCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding Foswiki? Send feedback