Using Fuzzy Logic and Q-Learning for Trust Modeling in Multi-agent Systems

• Autorzy: Abdullah Aref , Thomas Tran
• Języki publikacji: EN

Abstrakty

EN

Often in multi-agent systems, agents interact with other agents to fulfill their own goals. Trust is, therefore, considered essential to make such interactions effective. This work describes a trust model that augments fuzzy logic with Q-learning to help trust evaluating agents select beneficial trustees for interaction in uncertain, open, dynamic, and untrusted multi-agent systems. The performance of the proposed model is evaluated using simulation. The simulation results indicate that the proper augmentation of fuzzy subsystem to Q-learning can be useful for trust evaluating agents, and the resulting model can respond to dynamic changes in the environment.(original abstract)

Słowa kluczowe

EN

Fuzzy logic; Multi-agent system; Simulation

PL

Logika rozmyta; Systemy multiagentowe; Symulacja

• Czasopismo: Annals of Computer Science and Information Systems
• Rocznik: 2014
• Tom: 2
• Strony: 59--66

Twórcy

autor

Abdullah Aref

• University of Ottawa, Canada

autor

Thomas Tran

• University of Ottawa, Canada

Bibliografia

• Beldona S., "Reputation based buyer strategies for seller selection in electronic markets," Ph.D. dissertation, Electrical Engineering & Computer Science, University of Kansas, 2008.
• Burnett C., Norman T. J., and Sycara K., "Trust decision-making in multi-agent systems," in Proceedings of the Twenty-Second International Joint Conference on Artificial Intelligence - Volume Volume One, ser. IJCAI'11. AAAI Press, 2011.
• Cingolani P. and Alcala-Fdez J., "jfuzzylogic: a robust and flexible fuzzy-logic inference system language implementation," in Fuzzy Systems (FUZZ-IEEE), 2012 IEEE International Conference on, June 2012.
• Fullam K. K., Klos T. B., Muller G., Sabater J., Schlosser A., Topol Z., Barber K. S., Rosenschein J. S., Vercouter L., and Voss M., "A specification of the agent reputation and trust (art) testbed: Experimentation and competition for trust in agent societies," in Proceedings of the Fourth International Joint Conference on Autonomous Agents and Multiagent Systems, ser. AAMAS '05. New York, NY, USA: ACM, 2005.
• Georgoulas S., Moessner K., Mansour A., Pissarides M., and Spapis P., "A fuzzy reinforcement learning approach for pre-congestion notification based admission control," in Proceedings of the 6th IFIP WG 6.6 International Autonomous Infrastructure, Management, and Security Conference on Dependable Networks and Services, ser. AIMS'12. Berlin, Heidelberg: Springer-Verlag, 2013.
• Griffiths N., Chao K. -M., and Younas M., "Fuzzy trust for peer-to-peer systems," in Distributed Computing Systems Workshops, 2006. ICDCS Workshops 2006. 26th IEEE International Conference on, July 2006, pp. 73-73.
• Huynh T. D., Jennings N. R., and Shadbolt N. R., "An integrated trust and reputation model for open multi-agent systems," Autonomous Agents and Multi-Agent Systems, vol. 13, no. 2, pp. 119-154, Sep. 2006.
• Kaelbling L. P., Littman M. L., and Moore A. W., "Reinforcement learning: A survey," J. Artif. Int. Res., vol. 4, no. 1, pp. 237-285, May 1996.
• Kamvar S. D., Schlosser M. T., and Garcia-Molina H., "The eigentrust algorithm for reputation management in p2p networks," in Proceedings of the 12th International Conference on World Wide Web, ser. WWW '03. New York, NY, USA: ACM, 2003, pp. 640-651.
• Khosravifar B., Bentahar J., Gomrokchi M., and Alam R., "Crm: An efficient trust and reputation model for agent computing," Know.-Based Syst., vol. 30, pp. 1-16, Jun. 2012.
• Luke S., Cioffi-Revilla C., Panait L., Sullivan K., and Balan G., "Mason: A multiagent simulation environment," Simulation, vol. 81, no. 7, pp. 517-527, Jul. 2005.
• Mamdani E. and Assilian S., "An experiment in linguistic synthesis with a fuzzy logic controller," International Journal of Man-Machine Studies, vol. 7, no. 1, pp. 1 - 13, 1975.
• Mendel J., "Fuzzy logic systems for engineering: a tutorial," Proceedings of the IEEE, vol. 83, no. 3, pp. 345-377, Mar 1995.
• Pappis C. and Siettos C., "Fuzzy reasoning," in Search Methodologies, E. Burke and G. Kendall, Eds. Springer US, 2005, pp. 437-474.
• Pinyol I. and Sabater-Mir J., "Computational trust and reputation models for open multi-agent systems: A review," Artif. Intell. Rev., vol. 40, no. 1, pp. 1-25, Jun. 2013.
• Ramchurn S. D., Huynh D., and Jennings N. R., "Trust in multi-agent systems," Knowl. Eng. Rev., vol. 19, no. 1, pp. 1-25, Mar. 2004.
• Regan K. and Cohen R., "Indirect reputation assessment for adaptive buying agents in electronic markets," Business Agents and the Semantic Web workshop, vol. 1, 2005.
• Regan K., Cohen R., and Tran T., "Sharing models of sellers amongst buying agents in electronic marketplaces," Decentralized Agent Based and Social Approaches to User Modelling workshop, vol. 1, 2005.
• Song S., Hwang K., Zhou R., and Kwok Y.-K., "Trusted p2p transactions with fuzzy reputation aggregation," Internet Computing, IEEE, vol. 9, no. 6, pp. 24-34, Nov 2005.
• Sutton R. S. and Barto A. G., Introduction to Reinforcement Learning, 1st ed. Cambridge, MA, USA: MIT Press, 1998.
• Tran T. and Cohen R., "Improving user satisfaction in agent-based electronic marketplaces by reputation modelling and adjustable product quality," in Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems - Volume 2, ser. AAMAS '04. Washington, DC, USA: IEEE Computer Society, 2004, pp. 828-835.
• Tran T., "Protecting buying agents in e-marketplaces by direct experience trust modelling," Knowledge and Information Systems, vol. 22, no. 1, pp. 65-100, 2010.
• Yu H., Shen Z., Leung C., Miao C., and Lesser V., "A survey of multiagent trust management systems," Access, IEEE, vol. 1, pp. 35-50, 2013.