This is a discussion channel for “Symbol Emergence in Robotics: Pursuing Integrative Cognitive Architecture using Probabilistic Generative Models for Real-world Language Acquisition” by Prof. Tadahiro Taniguchi (Ritsumeikan University). A link to the talk is the following. Please do not share the link to anyone outside of this slack workspace. Access Passcode can be found at the announcement channel.

URL:https://vimeo.com/471278641 （28 minutes）

Let me share the slide I used in the presentation.

201011brain-ai_pub.pdf

Hi Tadahiro-san Thank you very much for your great talk about your challenges to connect symbol and signal worlds. As in the discussion yesterday, this topic would be essential to understand the human intelligence and to make close the baby-AI to the adult-AI. Your ‘probabilistic generative model for simultaneous acquisition of word units and multimodal object categories’ looks similar structure with the network for emotion representation talked by Yukie-san. Are these essentially the same from the viewpoint of learning? In the brain, emotional communication processing looks very different from the symbol-signal processing. My 2nd question is the structure of the network. The brain has different network structure for different kinds of processing, as mentioned in Kenji(Doya)-san’s presentation. Do you think that different unsupervised learning mechanisms or different network structures with a unified learning algorithm are required for the different purpose of computation, even if these all purposes can be interpreted as minimizing free-energy?

*Thread Reply:* Let me reply to your questions using two different threads.

*Thread Reply:* 1st one https://aibssymposium.slack.com/archives/C01BKSAGSKW/p1602385053383100

} 谷口忠大 (https://aibssymposium.slack.com/team/U01BYAVRKNZ)

<p>@Hiroaki Gomi &gt; Your ‘probabilistic generative model for simultaneous acquisition of word units and multimodal object categories’ looks similar structure with the network for emotion representation talked by Yukie-san. Are these essentially the same from the viewpoint of learning? In the brain, emotional communication processing looks very different from the symbol-signal processing. Basically, yes.</p> <p>To my understanding, her group used multimodal RBM for integrating multi-modal information for finding internal representation for emotion. It's bi-directional probabilistic model though Multimodal LDA and others we used are directed graphical models. But, I don't think it's a huge difference regarding the discussion over the point you raised.</p> <p>Abut connection of symbol and signal worlds, it is very important to differentiate "symbols" in symbolic logic which was used as internal representations in symbolic AI, which was propagated in cognitive science as well, and symbols, i.e., external representations, we humans use in our daily life. Symbols, i.e., signs in semiotics, in our daily communication is just a signals from this viewpoint. This is also why deep learning is so good at machine translation and wide range of NLP tasks without "symbol" manipulation, I think. Therefore, I don't think it's strange if the computational process of emotional communication processing and language processing looks similar.</p> <p><a href="https://aibssymposium.slack.com/archives/C01BKSAGSKW/p1602383599369100">https://aibssymposium.slack.com/archives/C01BKSAGSKW/p1602383599369100</a></p>

Original URL: https://aibssymposium.slack.com/archives/C01BKSAGSKW/p1602385053383100

Thread in <b>#s2_taniguchi</b>

*Thread Reply:* 2nd one https://aibssymposium.slack.com/archives/C01BKSAGSKW/p1602386997399800

} 谷口忠大 (https://aibssymposium.slack.com/team/U01BYAVRKNZ)

<p>@Hiroaki Gomi &gt; My 2nd question is the structure of the network. The brain has different network structure for different kinds of processing, as mentioned in Kenji(Doya)-san’s presentation. Do you think that different unsupervised learning mechanisms or different network structures with a unified learning algorithm are required for the different purpose of computation, even if these all purposes can be interpreted as minimizing free-energy? I'm not confident if I could get your point. But...</p> <p>Different modality, i.e., type of sensor information, has different latent structure embedded in the signal stream. Therefore, naturally, I think the network structure will be different. "Unsupervised learning mechanisms" can be divided into four parts, at least. (1) Structure of PGM, i.e., defining informational connections between variables, (2) approximators of each elemental distribution, e.g., neural network architecture in VAE, (3) inference procedure and (4) optimization criteria, i.e, target or principle.</p> <p>From the viewpoint of "algorithm," I don't think it should be unified. For example, for developing integrative cognitive systems for robots, we proposed Neuro-SERKET framework. That allows heterogeneous unsupervised learning modules to learn and work together. <code>Tadahiro Taniguchi, Tomoaki Nakamura, Masahiro Suzuki, Ryo Kuniyasu, Kaede Hayashi, Akira Taniguchi, Takato Horii, Takayuki Nagai, Neuro-SERKET: Development of Integrative Cognitive System through the Composition of Deep Probabilistic Generative Models, New Generation Computing, 84(), 2019.</code> <a href="https://arxiv.org/abs/1910.08918">https://arxiv.org/abs/1910.08918</a> &gt; even if these all purposes can be interpreted as minimizing free-energy? Free energy principle (FEP) explained in @Karl Friston’s talk is a quite convincing idea. I'm also willing to integrate such idea into our cognitive robotics studies. FEP is corresponding to (4) I mentioned above. (1) is mainly for integration of such different network structures. From the viewpoint of (2)-(3), I think different learning mechanism can be required for different functions or modules.</p>

Original URL: https://aibssymposium.slack.com/archives/C01BKSAGSKW/p1602386997399800

Posted in <b>#s2_taniguchi</b>

*Thread Reply:* @Hiroaki Gomi Thank you very much for your questions!

@Hiroaki Gomi > Your ‘probabilistic generative model for simultaneous acquisition of word units and multimodal object categories’ looks similar structure with the network for emotion representation talked by Yukie-san. Are these essentially the same from the viewpoint of learning? In the brain, emotional communication processing looks very different from the symbol-signal processing. Basically, yes.

To my understanding, her group used multimodal RBM for integrating multimodal information for finding internal representation for emotion. Although the Multimodal LDA, which we used, is a directed graphical model, it's a bi-directional probabilistic model. But, I don't think it's a huge difference regarding the discussion over the point you raised.

Abut connection of symbol and signal worlds; it is essential to differentiate "symbols" in symbolic logic, which was used as internal representations in symbolic AI and symbols, i.e., external representations, we humans use in our daily life. Symbols in our daily communication, i.e., signs in semiotics, are just signals from this viewpoint. This is also why deep learning is so good at machine translation and a wide range of NLP tasks without "symbol" manipulation, I think. Therefore, I don't think it's strange if the computational process of emotional communication processing and language processing looks similar.

https://aibssymposium.slack.com/archives/C01BKSAGSKW/p1602383599369100

*Thread Reply:* We have discussion over the difference between "symbols" in symbolic AI (and many discussions in AI), and symbols in our human society in the following paper. Tadahiro Taniguchi, Emre Ugur, Matej Hoffmann, Lorenzo Jamone, Takayuki Nagai, Benjamin Rosman, Toshihiko Matsuka, Naoto Iwahashi, Erhan Oztop, Justus Piater, Florentin Wörgötter, Symbol Emergence in Cognitive Developmental Systems: A Survey, IEEE Transactions on Cognitive and Developmental Systems, 11(4), pp.494-516, 2019. DOI: 10.1109/TCDS.2018.2867772 https://ieeexplore.ieee.org/document/8451965

*Thread Reply:* Here, as well. Tadahiro Taniguchi, Takayuki Nagai, Tomoaki Nakamura, Naoto Iwahashi, Tetsuya Ogata, and Hideki Asoh, Symbol Emergence in Robotics: A Survey, Advanced Robotics, 30, 11-12, 706-728, 2016. DOI:10.1080/01691864.2016.1164622 https://www.tandfonline.com/doi/abs/10.1080/01691864.2016.1164622#.Vw9iLfnNxVI

@Hiroaki Gomi > My 2nd question is the structure of the network. The brain has different network structure for different kinds of processing, as mentioned in Kenji(Doya)-san’s presentation. Do you think that different unsupervised learning mechanisms or different network structures with a unified learning algorithm are required for the different purpose of computation, even if these all purposes can be interpreted as minimizing free-energy? I'm not confident if I could get your point. But...

Different modality, i.e., type of sensor information, has a different latent structure embedded in the signal stream. Therefore, naturally, I think the network structure will be different. "Unsupervised learning mechanisms" can be divided into four parts, at least. (1) Structure of PGM, i.e., defining informational connections between variables, (2) approximators of each elemental distribution, e.g., neural network architecture in VAE, (3) inference procedure, and (4) optimization criteria, i.e., target or principle.

From the viewpoint of the "algorithm," I don't think it should be unified. For example, for developing integrative cognitive systems for robots, we proposed Neuro-SERKET framework. That allows heterogeneous unsupervised learning modules to learn and work together. Tadahiro Taniguchi, Tomoaki Nakamura, Masahiro Suzuki, Ryo Kuniyasu, Kaede Hayashi, Akira Taniguchi, Takato Horii, Takayuki Nagai, Neuro-SERKET: Development of Integrative Cognitive System through the Composition of Deep Probabilistic Generative Models, New Generation Computing, 84(), 2019. https://arxiv.org/abs/1910.08918 > even if these all purposes can be interpreted as minimizing free-energy? Free energy principle (FEP) explained in @Karl Friston’s talk is a quite convincing idea. I'm also willing to integrate such idea into our cognitive robotics studies. FEP is corresponding to (4) I mentioned above. (1) is mainly for the integration of such different network structures. From the viewpoint of (2)-(3), I think a different learning mechanism can be required for different functions or modules.