Zaawansowane procedury NLP jako przesłanka rekonstrukcji idei wiedzy

• Autorzy: Rafał Maciąg

Warianty tytułu

Advanced NLP Procedures as Premises for the Reconstruction of the Idea of Knowledge

• Języki publikacji: PL

Abstrakty

Przetwarzanie języka naturalnego (Natural Language Processing, NLP) należy do zespołu technologii określanych jako sztuczna inteligencja (AI). Jest to dziedzina bardzo obszerna, która w ostatnich latach przeżywa bezprecedensowy rozwój także w zakresie zastosowań praktycznych. W niniejszym tekście zostaną przedstawione przesłanki, które pozwalają zinterpretować obecne, najbardziej rozwinięte algorytmy NLP, w tym przede wszystkim model językowy GPT-3, jako istotne zmienne w interpretacji funkcji tekstu oraz, szerzej, problemu wiedzy. Ta ostatnia przechodzi istotną ewolucję rozumienia w wieku XX, tracąc swój czysto podmiotowy, tj. związany wyłącznie z człowiekiem, charakter, a także przestaje być traktowana jedynie jako przedmiot filozofii (epistemologii). (fragment tekstu)

EN

The article presents the current state of development of the Natural Language Processing (NLP) technology, in particular the GPT-3 language model, and presents its consequences for understanding the phenomenon of knowledge. The NLP technology has been experiencing remarkable development recently. The GPT-3 language model presents a level of advancement that allows it to generate texts as answers to general questions, as summaries of the presented text, etc., which reach the level surpassing the analogous level of human texts. These algorithmic operations lead to the determination of the probability distribution of its components. Texts generated by such a model should be considered as autonomous texts, using immanent, implicit knowledge embedded in language. This conclusion raises questions about the status of such knowledge. Help in the analysis is provided also by the theory of discourse, as well as the theory of discursive space based on it, that proposes the interpretation of knowledge as a trajectory of discourses in a dynamical space. Recognizing that knowledge may also be autonomous, and in particular not be at the exclusive disposal of humans, leads to the question of the status of artificial cognitive agents, such as the GPT-3 language model. (original abstract)

Słowa kluczowe

PL

Wiedza; Sztuczna inteligencja; Języki naturalne

EN

Knowledge; Artificial intelligence; Natural languages

• Czasopismo: Zarządzanie w Kulturze
• Rocznik: 2022
• Tom: 23
• Numer: z. 1
• Strony: 37--53

Twórcy

autor

Rafał Maciąg

• Uniwersytet Jagielloński w Krakowie

Bibliografia

• Aggarwal Charu C. (2018). Machine Learning for Text. Cham, Switzerland: Springer International Publishing.
• Angermuller Johannes, Maingueneau Dominique, Wodak Ruth (2014). An Introduction. W: Johannes Angermuller, Dominique Maingueneau, Ruth Wodak (red.), The Discourse Studies Reader: Main Currents in Theory and Analysis. Amsterdam-Philadelphia: John Benjamins Publishing Company, 1-14.
• Bender Emily M., Koller Alexander (2020). Climbing Towards NLU: On Meaning, Form, and Understanding in the Age of Data. W: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5185-5198, https://doi.org/10.18653/v1/2020. acl-main.463 [odczyt: 21.06.2022].
• Bleicher Joseph (1980). Contemporary Hermeneutics: Hermeneutics as Method, Philosophy and Critique (reprint, 1982 edition). London-Boston: Routledge & Kegan Paul.
• Brown Tom B., Mann Benjamin, Ryder Nick, Subbiah Melanie, Kaplan Jared, Dhariwal Prafulla, Neelakantan Arvind, Shyam Pranav, Sastry Girish, Askell Amanda, Agarwal Sandhini, Herbert-Voss Ariel, Krueger Gretchen, Henighan Tom, Child Rewon, Ramesh Aditya, Ziegler Daniel M., Wu Jeffrey, Winter Clemens, Hesse Christopher, Chen Mark, Sigler Eric, Litwin Mateusz, Gray Scott, Chess Benjamin, Clark Jack, Berner Christopher, McCandlish Sam, Radford Alec, Sutskever Ilya, Amodei Dario (2020). Language Models are Few-Shot Learners, ArXiv:2005.14165 [Cs], https://arxiv.org/ abs/2005.14165 [odczyt: 21.06.2022].
• Charniak Eugene (2019). Introduction to Deep Learning. Cambridge, Massachusetts: The MIT Press.
• Common Crawl Foundation (2021). Common Crawl, https://commoncrawl.org/ [odczyt: 21.06.2022].
• Crane Lesley (2016). Knowledge and Discourse Matters: Relocating Knowledge Management's Sphere of Interest onto Language. Hoboken, New Jersey: John Wiley & Sons, Inc.
• Deng Li, Liu Yang (red.) (2018). Deep Learning in Natural Language Processing. Singapore: Springer Singapore, https://doi.org/10.1007/978-981-10-5209-5 [odczyt: 21.06.2022].
• Descartes Rene (1637). Discours de la methode pour bien conduire sa raison, & chercher la verité dans les sciences. Plus La dioptrique. Les meteores. Et La geometrie. Qui sont des essais de cete methode. Leyde: De l'imprimerie de Ian Maire.
• Descartes Rene (2013). Rozprawa o metodzie. Właściwego kierowania rozumem i poszukiwania prawdy w naukach, tłum. Tadeusz Boy-Żeleński. Kęty: Wydawnictwo ANTYK.
• Dijk Teun Adrianus van (2013). Discourse and Knowledge. W: James Paul Gee, Michael Handford (red.), The Routledge Handbook of Discourse Analysis. London: Routledge, 587-603.
• Fairclough Norman (2003). Analysing Discourse: Textual Analysis for Social Research. London: Routledge.
• Flasiński Mariusz (2016). Introduction to Artificial Intelligence. Cham, Switzerland: Springer.
• Fortune Business Insights (2021). Natural Language Processing (NLP) Market Size & Growth, 2028, https://www.fortunebusinessinsights.com/industry-reports/natural-language-processing- nlp-market-101933 [odczyt: 21.06.2022].
• Foucault Michel (1966). Les mots et les choses: Une archéologie des sciences humaines. Paris: Gallimard.
• Foucault Michel (1969). L'archéologie du savoir. Paris: Gallimard.
• Foucault Michel (1971). L'ordre du discours: Leçon inaugurale au College de France prononcée le 2 décembre 1970. Paris: Gallimard.
• Foucault Michel (1981). The Order of Discourse. W: Robert E. Young (red.), Untying the Text: A Post-Structuralist Reader. Boston: Routledge & Kegan Paul, 51-78.
• Goodfellow Ian, Bengio Yoshua, Courville Aaron (2016). Deep Learning. Cambridge, Massachusetts: MIT Press, https://www.deeplearningbook.org [odczyt: 21.06.2022].
• Helwe Chadi, Clavel Chloé, Suchanek Fabian M. (2021). Reasoning with Transformer-based Models: Deep Learning, but Shallow Reasoning, 3rd Conference on Automated Knowledge Base Construction, June 22, https://openreview.net/forum?id=Ozp1WrgtF5_ [odczyt: 21.06.2022].
• Hochreiter Sepp, Schmidhuber Jürgen (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780, https://doi.org/10.1162/neco.1997.9.8.1735 [odczyt: 21.06.2022].
• Hyland Ken, Paltridge Brian (red.) (2011). Bloomsbury Companion to Discourse Analysis. London- New York: Bloomsbury.
• Jorgensen Marianne, Phillips Louise (2002). Discourse Analysis as Theory and Method. London- Thousand Oaks, California: Sage Publications.
• Jurafsky Dan, Martin James H. (2020). Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, 3rd ed. Upper Saddle River, New Jersey: Prentice Hall.
• Kozlowski Austin C., Taddy Matt, Evans James A. (2019). The Geometry of Culture: Analyzing Meaning through Word Embeddings. American Sociological Review, 84(5), 905-949, https://doi.org/10.1177/0003122419877135 [odczyt: 21.06.2022].
• Lévy Pierre (1999). Collective Intelligence, Mankind's Emerging World in Cyberspace. Cambridge, Massachusetts: Perseus Books.
• Lyotard Jean-François (1979). La condition postmoderne: Rapport sur le savoir. Paris: Éd. de Minuit.
• Maciąg Rafał (2018). Discursive Space and Its Consequences for Understanding Knowledge and Information. Philosophies, 3(4), https://doi.org/10.3390/philosophies3040034 [odczyt: 21.06.2022].
• Maciąg Rafał (2020). Ontological Basis of Knowledge in the Theory of Discursive Space and Its Consequences. Proceedings, 47(1), 11, https://doi.org/10.3390/proceedings47010011 [odczyt: 21.06.2022].
• Maciąg Rafał (2022). Wiedza jako opowieść. Przestrzeń dyskursywna. Kraków: TAiWPN Universitas.
• Malpas Jeff, Gander Hans-Helmut (red.) (2015). The Routledge Companion to Hermeneutics. Abingdon, Oxon: Routledge.
• Merrill William, Goldberg Yoav, Schwartz Roy, Smith Noah A. (2021). Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand? Transactions of the Association for Computational Linguistics, 9, 1047-1060, https://doi.org/10.1162/tacl_a_00412 [odczyt: 21.06.2022].
• Mikolov Tomáš, Sutskever Ilya, Chen Kai, Corrado Greg, Dean Jeffrey (2013). Distributed Representations of Words and Phrases and their Compositionality, ArXiv:1310.4546 [Cs, Stat], https://arxiv.org/abs/1310.4546 [odczyt: 21.06.2022].
• Neelakantan Arvind, Xu Tao, Puri Raul, Radford Alec, Han Jesse Michael, Tworek Jerry, Yuan Qiming, Tezak Nikolas, Kim Jong Wook, Hallacy Chris, Heidecke Johannes, Shyam Pranav, Power Boris, Nekoul Tyna Eloundou, Sastry Girish, Krueger Gretchen, Schnurr David, Such Felipe Petroski, Hsu Kenny, Thompson Madeleine, Khan Tabarak, Sherbakov Toki, Jang Joanne, Welinder Peter, Weng Lilian (2022). Text and Code Embeddings by Contrastive Pre-Training, ArXiv:2201.10005 [Cs], https://arxiv.org/abs/2201.10005 [odczyt: 21.06.2022].
• Ouyang Long, Wu Jeff, Jiang Xu, Almeida Diogo, Wainwright Carroll L., Mishkin Pamela, Zhang Chong, Agarwal Sandhini, Slama Katarina, Ray Alex, Schulman John, Hilton Jacob, Kelton Fraser, Miller Luke, Simens Maddie, Askell Amanda, Welinder Peter, Christiano Paul, Leike Jan, Lowe Ryan (2022). Training Language Models to Follow Instructions with Human Feedback, https://arxiv.org/abs/2203.02155 [odczyt: 21.06.2022].
• Paltridge Brian (2006). Discourse Analysis: An Introduction. London-New York: Continuum.
• Petroni Fabio, Rocktäschel Tim, Lewis Patrick, Bakhtin Anton, Wu Yuxiang, Miller Alexander H., Riedel Sebastian (2019). Language Models as Knowledge Bases? W: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), 2463-2473, https:// doi.org/10.18653/v1/D19-1250 [odczyt: 21.06.2022].
• Porada Ian, Sordoni Alessandro, Cheung Jackie C.K. (2021). Does Pre-training Induce Systematic Inference? How Masked Language Models Acquire Commonsense Knowledge, ArXiv:2112.08583 [Cs], https://arxiv.org/abs/2112.08583 [odczyt: 21.06.2022].
• Porter Stanley E., Robinson Jason C. (2011). Hermeneutics: An Introduction to Interpretative Theory. Grand Rapids, Michigan-Cambridge, U.K.: William B. Eerdmans Publishing Company.
• Qiu Wenjun, Xu Yang (2022). HistBERT: A Pre-trained Language Model for Diachronic Lexical Semantic Analysis, ArXiv:2202.03612 [Cs], https://doi.org/10.13140/RG.2.2.14905.44649 [odczyt: 21.06.2022].
• Radford Alec, Wu Jeffrey, Child Rewon, Luan David, Amodei Dario, Sutskever Ilya (2019). Language Models are Unsupervised Multitask Learners, https://d4mucfpksywv.cloudfront. net/better-language-models/language_models_are_unsupervised_multitask_learners.pdf [odczyt: 21.06.2022].
• Rajpurkar Pranav, Jia Robin, Liang Percy (2018). Know What You Don't Know: Unanswerable Questions for SQuAD, ArXiv:1806.03822 [Cs], https://arxiv.org/abs/1806.03822 [odczyt: 21.06.2022].
• Rajpurkar Pranav, Zhang Jian, Lopyrev Konstantin, Liang Percy (2016). SQuAD: 100, 000+ Questions for Machine Comprehension of Text, CoRR, ArXiv:1606.05250, https://arxiv.org/ abs/1606.05250 [odczyt: 21.06.2022].
• Safavi Tara, Koutra Danai (2021). Relational World Knowledge Representation in Contextual Language Models: A Review, ArXiv:2104.05837 [Cs], https://arxiv.org/abs/2104.05837 [odczyt: 21.06.2022].
• Sowa John F. (2010). The Role of Logic and Ontology in Language and Reasoning. W: Roberto Poli, Johanna Seibt (red.), Theory and Applications of Ontology: Philosophical Perspectives. Dordrecht: Springer Netherlands, 231-263, https://doi.org/10.1007/978-90-481-8845-1_11 [odczyt: 21.06.2022].
• Tamkin Alex, Brundage Miles, Clark Jack, Ganguli Deep (2021). Understanding the Capabilities, Limitations, and Societal Impact of Large Language Models, ArXiv:2102.02503, https://arxiv. org/abs/2102.02503v1 [odczyt: 21.06.2022].
• Traylor Aaron, Feiman Roman, Pavlick Ellie (2021). AND does not Mean OR: Using Formal Languages to Study Language Models' Representations. W: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing. Vol. 2: Short papers, 158-167, https://doi. org/10.18653/v1/2021.acl-short.21 [odczyt: 21.06.2022].
• Turing Alan M. (1950). Computing Machinery and Intelligence. Mind, 236(14), 433-460.
• Vaswani Ashish, Shazeer Noam, Parmar Niki, Uszkoreit Jakob, Jones Llion, Gomez Aidan N., Kaiser Lukasz, Polosukhin Illia (2017). Attention Is All You Need, ArXiv:1706.03762 [Cs], https://arxiv.org/abs/1706.03762 [odczyt: 21.06.2022].
• Wang Alex, Singh Amanpreet, Michael Julian, Hill Felix, Levy Omer, Bowman Ssamuel R. (2018). GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding, CoRR, ArXiv:1804.07461, https://arxiv.org/abs/1804.07461 [odczyt: 21.06.2022].
• Wittgenstein Ludwig (1958). Philosophical Investigations (second edition). Oxford: Basil Blackwell. Yoshida Ryo, Oseki Yohei (2022). Learning Argument Structures with Recurrent Neural Network
• Grammars. W: Proceedings of the Society for Computation in Linguistics, 5(1), 101-111, https://doi.org/10.7275/kne0-hc86 [odczyt: 21.06.2022].
• Zhang Daniel, Maslej Nestor, Brynjolfsson Erik, Etchemendy John, Lyons Terah, Manyika James, Ngo Helen, Niebles Juan C., Sellitto Michael, Sakhaee Ellie, Shoham Yoav, Jack Clark, Perrault Raymond (2021). The AI Index 2021 Annual Report, Stanford, California: AI Index Steering Committee, Human-Centered AI Institute, Stanford University.
• Zhou Xuhui, Zhang Yue, Cui Leyang, Huang Dandan (2020). Evaluating Commonsense in Pre-Trained Language Models. W: Proceedings of the AAAI Conference on Artificial Intelligence, 34(05), 9733-9740, https://doi.org/10.1609/aaai.v34i05.6523 [odczyt: 21.06.2022].

Identyfikatory

DOI

10.4467/20843976ZK.22.003.15869