多主体强化学习协作策略研究

Chapter 1  Introduction1.1  Reinforcement Learning1.1.1  Generality of Reinforcement Learning1.1.2  Reinforcement Learning on Markov Decision Processes1.1.3  Integrating Reinforcement Learning into Agent Architecture1.2  Multiagent Reinforcement Learning1.2.1  Multiagent Systems1.2.2  Reinforcement Learning in Multiagent Systems1.2.3  Learning and Coordination in Multiagent Systems1.3  Ant System for Stochastic Combinatorial Optimization1.3.1  Ants Forage Behavior1.3.2  Ant Colony Optimization1.3.3  MAX-MIN Ant System1.4  Motivations and Consequences1.5  Book SummaryBibliographyChapter 2  Reinforcement Learning and Its Combination with Ant Colony System2.1  Introduction2.2  Investigation into Reinforcement Learning and Swarm Intelligence2.2.1  Temporal Differences Learning Method2.2.2  Active Exploration and Experience Replay in Reinforcement Learning2.2.3  Ant Colony System for Traveling Salesman Problem2.3  The Q-ACS Multiagent Learning Method2.3.1  The Q-ACS Learning Algorithm2.3.2  Some Properties of the Q-ACS Learning Method2.3.3  Relation with Ant-Q Learning Method2.4  Simulations and Results2.5  ConclusionsBibliographyChapter 3  Multiagent Learning Methods Based on Indirect Media Information Sharing3.1  Introduction3.2  The Multiagent Learning Method Considering Statistics Features3.2.1  Accelerated K-certainty Exploration3.2.2  The T-ACS Learning Algorithm3.3  The Heterogeneous Agents Learning3.3.1  The D-ACS Learning Algorithm3.3.2  Some Discussions about the D-ACS Learning Algorithm3.4  Comparisons with Related State-of-the-arts3.5  Simulations and Results3.5.1  Experimental Results on Hunter Game3.5.2  Experimental Results on Traveling Salesman Problem3.6  ConclusionsBibliographyChapter 4  Action Conversion Mechanism in Multiagent Reinforcement Learning4.1  Introduction4.2  Model-Based Reinforcement Learning4.2.1  Dyna-Q Architecture4.2.2  Prioritized Sweeping Method4.2.3  Minimax Search and Reinforcement Learning4.2.4  RTP-Q Learning4.3  The Q-ac Multiagent Reinforcement Learning4.3.1  Task Model4.3.2  Converting Action4.3.3  Multiagent Cooperation Methods4.3.4  Q-value Update4.3.5  The Q-ac Learning Algorithm4.3.6  Using Adversarial Action Instead o{ ~ Probability Exploration4.4  Simulations and Results4.5  ConclusionsBibliographyChapter 5  Multiagent Learning Approaches Applied to Vehicle Routing Problems5.1  Introduction5.2  Related State-of-the-arts5.2.1  Some Heuristic Algorithms5.2.2  The Vehicle Routing Problem with Time Windows5.3  The Multiagent Learning Applied to CVRP and VRPTW5.4  Simulations and Results5.5  ConclusionsBibliographyChapter 6  Multiagent learning Methods Applied to Multicast Routing Problems6.1  Introduction6.2  Multiagent Q-learning Applied to the Network Routing6.2.1  Investigation into Q-routing6.2.2  AntNet Investigation6.3  Some Multicast Routing in Mobile Ad Hoc Networks6.4  The Multiagent Q-learning in the Q-MAP Multicast Routing Method6.4.1  Overview of the Q-MAP Multicast Routing6.4.2  Join Query Packet, Join Reply Packet and Membership Maintenance6.4.3  Convergence Proof of Q-MAP Method6.5  Simulations and Results6.6  ConclusionsBibliographyChapter 7  Multiagent Reinforcement Learning for Supply Chain Management7.1  Introduction7.2  Related Issues of Supply Chain Management7.3  SCM Network Scheme with Multiagent Reinforcement Learning7.3.1  SCM with Multiagent7.3.2  The RL Agents in SCM Network7.4  Application of the Q-ACS Method to SCM7.4.1  The Application Model in SCM7.4.2  The Q-ACS Learning Applied to the SCM System7.5  ConclusionBibliographyChapter 8  Multiagent Learning Applied in Supply Chain Ordering Management8.1  Introduction8.2  Supply Chain Management Model8.3  The Multiagent Learning Model for SC Ordering Management8.4  Simulations and Results8.5  ConclusionsBibliography