f-leno / adhoc_aamas-17 Goto Github PK

Codification used for the AAMAS-17 paper "Simultaneously Learning and Advising in Multiagent Reinforcement Learning"

Jupyter Notebook 69.15% Makefile 21.87% Python 0.68% Shell 0.15% C++ 6.59% C 0.29% CMake 1.12% TeX 0.14%

adhoc_aamas-17's Introduction

Simultaneously Learning and Advising in Multiagent Reinforcement Learning

This is the codification used in the AAMAS 2017 paper proposing Ad Hoc Advising as means of accelerating learning in Multiagent Systems composed of simultaneously learning agents. You are free to use all or part of the codes here presented for any purpose, provided that the paper is properly cited and the original authors properly credited. All the files here shared come with no warranties.

Paper bib entry:

@inproceedings{SilvaAndCosta2017,
author = {Silva, Felipe Leno da and
Ruben Glatt and
Anna Helena Reali Costa},
title = {{Simultaneously Learning and Advising in Multiagent Reinforcement Learning}},
booktitle = {Proceedings of the 16th International Conference on Autonomous Agents and Multiagent Systems (AAMAS)},
year = {2017},
pages = {1100--1108}
}

This project was built on Python 2.7. All the experiments are executed in the HFO platform (https://github.com/LARG/HFO), we included the version we used in the HFO folder (slighly different from the standard HFO). For the graph generation code you will need to install Jupyter Notebook (http://jupyter.readthedocs.io/en/latest/install.html).

Files

The folder HFO contains the HFO server we used for experiments.

The folder AdHoc contains our implementation of all algorithms and experiments.

Finally, the folder ProcessedFiles contains already processed .csv files for graph printing and data visualization.

How to use

First install HFO following instructions in https://github.com/LARG/HFO.

In folder AdHoc, executing the script experiment1and2.sh is enough to run the first and second experiment. However, it will take a very long time until the experiments are completed. It may be of interest running more than one algorithm at the same time if you have enough computing power.

Executing experiment3.sh runs the third experiment. Before running this experiment, the script pretrain.sh should be executed, so as to store the Q-table for the already trained agent.

The result of any experiment is a folder with .csv files, that can be used to generate graphs using evaluation-leno.ipynb in jupyter notebook. (all the files used for the paper are in the folder ProcessedFiles).

Contact

For questions about the Codification or paper, please send an email to the first author.

adhoc_aamas-17's People

Contributors

Stargazers

Watchers

Forkers

yupenggao bluecontra wwxfromtju jloveu cheryyunl wangxinnevergiveup amrit-pal-singh tamurajunichi wangwei-97 lorenzflow

adhoc_aamas-17's Issues

ImportError: No module named cmac

when I try:
from cmac import CMAC
I got error:

ImportError:No module named cmac

and I use:
pip install cmac
and,

Could not find a version that satisfies the requirement cmac (from versions: )
No matching distribution found for cmac

Segmentation fault (core dumped)

Dear leno,
When I installed your built-in HFO environment, I couldn't run experiment, so I used the original HFO environment and was able to run experiment and did normal training and evaluation. However, the 50 rounds of training usually only run for 2 or 3 rounds before I get a "Segmentation fault (core dumped)" error. Is this caused by using the original HFO environment?
I checked the exact location of the Segmentation fault:

  File "/home/iccd/anaconda3/envs/HFO/lib/python2.7/site-packages/hfo/hfo.py", line 179 in step
  File "../AdHoc_AAMAS-17-master/AdHoc/agents/sarsa.py", line 105 in step
  File "../AdHoc_AAMAS-17-master/AdHoc/agents/adhoc.py", line 149 in step
  File "experiment.py", line 175 in thread_agent
  File "/home/iccd/anaconda3/envs/HFO/lib/python2.7/threading.py", line 754 in run
  File "/home/iccd/anaconda3/envs/HFO/lib/python2.7/threading.py", line 801 in __bootstrap_inner
  File "/home/iccd/anaconda3/envs/HFO/lib/python2.7/threading.py", line 774 in __bootstrap

Here is the logs when I'm trying to run experiment1and2.sh(wrong part)、

Namespace(agent1='AdHocTD', agent10='Dummy', agent11='Dummy', agent2='AdHocTD', agent3='AdHocTD', agent4='Dummy', agent5='Dummy', agent6='Dummy', agent7='Dummy', agent8='Dummy', agent9='Dummy', evaluation_duration=100, evaluation_interval=20, learning_trials=5000, log_file='../log3/', number_agents=3, number_trial=4, port=10000, seed=12345, server_path='../HFO/bin/')
***** Loading agent implementations
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
here is AdHocTD init
OK Agent
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
here is AdHocTD init
OK Agent
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
here is AdHocTD init
OK Agent
***** %s: Agents online --> %s
Agent Classes OK
number_trial: 4
this is number 0 agent
number_trial:4
******Connecting agent 0****
***** Connecting to HFO server on port 10000
******************************************************************
 librcsc 4.1.0
 Copyright 2000 - 2007. Hidehisa Akiyama.
 Copyright 2007 - 2011. Hidehisa Akiyama and Hiroki Shimora
 All rights reserved.
******************************************************************
this is number 1 agent
number_trial:4
******Connecting agent 1****
***** Connecting to HFO server on port 10000
******************************************************************
 librcsc 4.1.0
 Copyright 2000 - 2007. Hidehisa Akiyama.
 Copyright 2007 - 2011. Hidehisa Akiyama and Hiroki Shimora
 All rights reserved.
******************************************************************
 -1: waited 5 seconds. server down??

agent suddenly can not connect to sever

ERROR Failed to read team strategy. Init failed

Hi @f-leno
I am trying to run the code, but getting

***ERROR*** Failed to read team strategy.
Init failed

Here is the logs when I'm trying to run experiment1and2.sh

Namespace(agent1='AdHocTD', agent10='Dummy', agent11='Dummy', agent2='AdHocTD', agent3='AdHocTD', agent4='Dummy', agent5='Dummy', agent6='Dummy', agent7='Dummy', agent8='Dummy', agent9='Dummy', evaluation_duration=100, evaluation_interval=20, learning_trials=5000, log_file='../log/', number_agents=3, number_trial=1, port=20000, seed=12345, server_path='../HFO/bin/')
***** Loading agent implementations
Parsing Paaass
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
OK Agent
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
OK Agent
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
OK Agent
***** %s: Agents online --> %s
Agent Classes OK
******Connecting agent 0****
***** Connecting to HFO server on port 20000
******************************************************************
 librcsc 4.1.0
 Copyright 2000 - 2007. Hidehisa Akiyama.
 Copyright 2007 - 2011. Hidehisa Akiyama and Hiroki Shimora
 All rights reserved.
******************************************************************
base_left 7: [0, 7]  see synch.
******Connecting agent 1****
***** Connecting to HFO server on port 20000
******************************************************************
 librcsc 4.1.0
 Copyright 2000 - 2007. Hidehisa Akiyama.
 Copyright 2007 - 2011. Hidehisa Akiyama and Hiroki Shimora
 All rights reserved.
******************************************************************
Namespace(agent1='AdHocTD', agent10='Dummy', agent11='Dummy', agent2='AdHocTD', agent3='AdHocTD', agent4='Dummy', agent5='Dummy', agent6='Dummy', agent7='Dummy', agent8='Dummy', agent9='Dummy', evaluation_duration=100, evaluation_interval=20, learning_trials=5000, log_file='../log/', number_agents=3, number_trial=2, port=20000, seed=12345, server_path='../HFO/bin/')
***** Loading agent implementations
Parsing Paaass
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
OK Agent
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
OK Agent
AgentName: AdHocTD
Creating agent
Param: 0.9 - 0.5
OK Agent
***** %s: Agents online --> %s
Agent Classes OK
******Connecting agent 0****
***** Connecting to HFO server on port 20000
******************************************************************
 librcsc 4.1.0
 Copyright 2000 - 2007. Hidehisa Akiyama.
 Copyright 2007 - 2011. Hidehisa Akiyama and Hiroki Shimora
 All rights reserved.
******************************************************************
base_left 8: [0, 6]  see synch.
******Connecting agent 1****
***** Connecting to HFO server on port 20000
******************************************************************
 librcsc 4.1.0
 Copyright 2000 - 2007. Hidehisa Akiyama.
 Copyright 2007 - 2011. Hidehisa Akiyama and Hiroki Shimora
 All rights reserved.

Agents are trying to connect but are unable to connect successfully.

f-leno / adhoc_aamas-17 Goto Github PK

adhoc_aamas-17's Introduction

Simultaneously Learning and Advising in Multiagent Reinforcement Learning

Files

How to use

Contact

adhoc_aamas-17's People

Contributors

Stargazers

Watchers

Forkers

adhoc_aamas-17's Issues

ImportError: No module named cmac

Segmentation fault (core dumped)

ERROR Failed to read team strategy. Init failed

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent