Git Product home page Git Product logo

vdptag2.jl's Introduction

VDPTag2

Build Status

Coverage Status

codecov.io

See VDPTag2/test for usage examples.

VDPTag2 with POMCPOW.jl

using POMDPs
using POMCPOW
using POMDPModels
using POMDPSimulators
using POMDPPolicies
using VDPTag2

solver = POMCPOWSolver(criterion=MaxUCB(20.0))
pomdp = VDPTagPOMDP() # from VDPTag2
planner = solve(solver, pomdp)

hr = HistoryRecorder(max_steps=100)
hist = simulate(hr, pomdp, planner)
for (s, b, a, r, sp, o) in hist
    @show s, a, r, sp
end

rhist = simulate(hr, pomdp, RandomPolicy(pomdp))
println("""
    Cumulative Discounted Reward (for 1 simulation)
        Random: $(discounted_reward(rhist))
        POMCPOW: $(discounted_reward(hist))
    """)

VDPTag2 solved via POMCPOW.jl.

vdptag2.jl's People

Contributors

chandlerkilpatrick1 avatar jamgochiana avatar leeyiyuan avatar logankilpatrick avatar whifflefish avatar zsunberg avatar

Stargazers

avatar avatar avatar avatar avatar avatar

Watchers

avatar avatar avatar

Forkers

chandlerkilpatrick1 lamda-pomdp whifflefish wqy-julia jamgochiana

vdptag2.jl's Issues

Error Generating Gif

Hey @zsunberg I am trying to get gif.jl to run so we can have the visuals working. Here is the error I am getting in Julia 0.7

ERROR: LoadError: No user recipe defined for VDPTagPOMDP{CardinalBarriers}
Stacktrace:
 [1] error(::String) at ./error.jl:33
 [2] convertToAnyVector(::VDPTagPOMDP{CardinalBarriers}, ::Dict{Symbol,Any}) at /Users/chandlerkilpatrick/.julia/packages/Plots/ex9On/src/series.jl:14
 [3] macro expansion at /Users/chandlerkilpatrick/.julia/packages/Plots/ex9On/src/series.jl:131 [inlined]
 [4] apply_recipe(::Dict{Symbol,Any}, ::Type{Plots.SliceIt}, ::VDPTagPOMDP{CardinalBarriers}, ::POMDPSimulators.SubPOMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState},POMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState}},UnitRange{Int64}}, ::Nothing) at /Users/chandlerkilpatrick/.julia/packages/RecipesBase/Sk8IO/src/RecipesBase.jl:293
 [5] _process_userrecipes(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{VDPTagPOMDP{CardinalBarriers},POMDPSimulators.SubPOMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState},POMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState}},UnitRange{Int64}}}) at /Users/chandlerkilpatrick/.julia/packages/Plots/ex9On/src/pipeline.jl:83
 [6] macro expansion at ./logging.jl:307 [inlined]
 [7] _plot!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{VDPTagPOMDP{CardinalBarriers},POMDPSimulators.SubPOMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState},POMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState}},UnitRange{Int64}}}) at /Users/chandlerkilpatrick/.julia/packages/Plots/ex9On/src/plot.jl:171
 [8] #plot#136(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::VDPTagPOMDP{CardinalBarriers}, ::Vararg{Any,N} where N) at /Users/chandlerkilpatrick/.julia/packages/Plots/ex9On/src/plot.jl:57
 [9] plot(::VDPTagPOMDP{CardinalBarriers}, ::POMDPSimulators.SubPOMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState},POMDPHistory{TagState,TagAction,SArray{Tuple{8},Float64,1,8},ParticleCollection{TagState}},UnitRange{Int64}}) at /Users/chandlerkilpatrick/.julia/packages/Plots/ex9On/src/plot.jl:51
 [10] macro expansion at /Users/chandlerkilpatrick/.julia/dev/VDPTag2/test/gif.jl:29 [inlined]
 [11] top-level scope at /Users/chandlerkilpatrick/.julia/packages/ProgressMeter/TpeOV/src/ProgressMeter.jl:599
 [12] include at ./boot.jl:317 [inlined]
 [13] include_relative(::Module, ::String) at ./loading.jl:1038
 [14] include(::Module, ::String) at ./sysimg.jl:29
 [15] include(::String) at ./client.jl:398
 [16] top-level scope at none:0
in expression starting at /Users/chandlerkilpatrick/.julia/dev/VDPTag2/test/gif.jl:28

And this is the error in Julia 1.2

ERROR: LoadError: MethodError: no method matching lastindex(::Base.Generator{SubArray{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}},1,Array{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}},1},Tuple{UnitRange{Int64}},true},getfield(POMDPSimulators, Symbol("##6#7")){Symbol}})
Closest candidates are:
  lastindex(::Core.SimpleVector) at essentials.jl:603
  lastindex(::Cmd) at process.jl:963
  lastindex(::Base64.Buffer) at /Users/sabae/buildbot/worker/package_macos64/build/usr/share/julia/stdlib/v1.2/Base64/src/buffer.jl:19
  ...
Stacktrace:
 [1] macro expansion at /Users/chandlerkilpatrick/.julia/dev/VDPTag2/src/visualization.jl:53 [inlined]
 [2] macro expansion at /Users/chandlerkilpatrick/.julia/packages/RecipesBase/zBoFG/src/RecipesBase.jl:312 [inlined]
 [3] macro expansion at /Users/chandlerkilpatrick/.julia/dev/VDPTag2/src/visualization.jl:52 [inlined]
 [4] apply_recipe(::Dict{Symbol,Any}, ::VDPTagMDP{CardinalBarriers}, ::POMDPSimulators.SubHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}},SimHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}}},UnitRange{Int64}}) at /Users/chandlerkilpatrick/.julia/packages/RecipesBase/zBoFG/src/RecipesBase.jl:275
 [5] _process_userrecipes(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{VDPTagPOMDP{CardinalBarriers},POMDPSimulators.SubHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}},SimHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}}},UnitRange{Int64}}}) at /Users/chandlerkilpatrick/.julia/packages/Plots/Iuc9S/src/pipeline.jl:83
 [6] _plot!(::Plots.Plot{Plots.GRBackend}, ::Dict{Symbol,Any}, ::Tuple{VDPTagPOMDP{CardinalBarriers},POMDPSimulators.SubHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}},SimHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}}},UnitRange{Int64}}}) at /Users/chandlerkilpatrick/.julia/packages/Plots/Iuc9S/src/plot.jl:178
 [7] #plot#137(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::typeof(plot), ::VDPTagPOMDP{CardinalBarriers}, ::Vararg{Any,N} where N) at /Users/chandlerkilpatrick/.julia/packages/Plots/Iuc9S/src/plot.jl:57
 [8] plot(::VDPTagPOMDP{CardinalBarriers}, ::POMDPSimulators.SubHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}},SimHistory{NamedTuple{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),Tuple{TagState,TagAction,TagState,SArray{Tuple{8},Float64,1,8},Float64,Int64,Nothing,ParticleCollection{TagState},ParticleCollection{TagState},Nothing}}},UnitRange{Int64}}) at /Users/chandlerkilpatrick/.julia/packages/Plots/Iuc9S/src/plot.jl:51
 [9] macro expansion at /Users/chandlerkilpatrick/.julia/dev/VDPTag2/test/gif.jl:29 [inlined]
 [10] top-level scope at /Users/chandlerkilpatrick/.julia/packages/ProgressMeter/g1lse/src/ProgressMeter.jl:717
 [11] include at ./boot.jl:328 [inlined]
 [12] include_relative(::Module, ::String) at ./loading.jl:1094
 [13] include(::Module, ::String) at ./Base.jl:31
 [14] include(::String) at ./client.jl:431
 [15] top-level scope at REPL[1]:1
in expression starting at /Users/chandlerkilpatrick/.julia/dev/VDPTag2/test/gif.jl:28

Quick question about action spaces

Hey Zach, when this code is run:

struct POVDPTagActionSpace end
rand(rng::AbstractRNG, ::POVDPTagActionSpace) = TagAction(rand(rng, Bool), 2*pi*rand(rng))
POMDPs.actions(::VDPTagPOMDP) = POVDPTagActionSpace()

How is it actually creating the action space? It looks like there would only be a single action here?

POMCPOW in VDPTag

Hey @zsunberg, could you explain where POMCPOW is integrated into VDPTag2? I'm having trouble figuring out where the solver is being defined inside VDPTag2.

Where is the rand(rng, Bool) function defined?

Hey Zach, where is the rand function defined for the 2nd rand call in this function:

VDPTag2.jl/src/VDPTag2.jl

Line 123 in 9e219a0

rand(rng::AbstractRNG, ::POVDPTagActionSpace) = TagAction(rand(rng, Bool), 2*pi*rand(rng))
?

I don't see where rand(rng, Bool) is defined anywhere. I am getting an error saying that the Sampler for that object is not defined but I don't know where to wrap that function since I don't see it in the code.

Can't apply to BasicPOMCP.jl

The code is as follows:

using POMDPs
using BasicPOMCP
using POMDPModels
using POMDPSimulators
using POMDPPolicies
using VDPTag2

solver = POMCPSolver()
pomdp = VDPTagPOMDP() # from VDPTag2
planner = solve(solver, pomdp)

hr = HistoryRecorder(max_steps=100)
hist = simulate(hr, pomdp, planner)
for (s, b, a, r, sp, o) in hist
    @show s, a, r, sp
end

rhist = simulate(hr, pomdp, RandomPolicy(pomdp))
println("""
    Cumulative Discounted Reward (for 1 simulation)
        Random: $(discounted_reward(rhist))
        POMCPOW: $(discounted_reward(hist))
    """)

And the error message says:

ERROR: LoadError: MethodError: no method matching length(::VDPTag2.POVDPTagActionSpace)
Closest candidates are:
length(!Matched::Core.SimpleVector) at essentials.jl:593
length(!Matched::Base.MethodList) at reflection.jl:849
length(!Matched::Core.MethodTable) at reflection.jl:923
...
Stacktrace:
[1] _similar_for(::UnitRange{Int64}, ::Type{Any}, ::VDPTag2.POVDPTagActionSpace, ::Base.HasLength) at ./array.jl:533
[2] _collect(::UnitRange{Int64}, ::VDPTag2.POVDPTagActionSpace, ::Base.HasEltype, ::Base.HasLength) at ./array.jl:566
[3] collect(::VDPTag2.POVDPTagActionSpace) at ./array.jl:560
[4] BasicPOMCP.POMCPTree(::VDPTagPOMDP{Nothing}, ::ParticleFilters.ParticleCollection{TagState}, ::Int64) at /home/fengliu/.julia/packages/BasicPOMCP/mKBSS/src/BasicPOMCP.jl:132
[5] #action_info#4(::Bool, ::typeof(POMDPModelTools.action_info), ::POMCPPlanner{VDPTagPOMDP{Nothing},BasicPOMCP.SolvedPORollout{RandomPolicy{Random._GLOBAL_RNG,VDPTagPOMDP{Nothing},BeliefUpdaters.NothingUpdater},BeliefUpdaters.NothingUpdater,Random._GLOBAL_RNG},Random._GLOBAL_RNG}, ::ParticleFilters.ParticleCollection{TagState}) at /home/fengliu/.julia/packages/BasicPOMCP/mKBSS/src/solver.jl:5
[6] action_info at /home/fengliu/.julia/packages/BasicPOMCP/mKBSS/src/solver.jl:2 [inlined]
[7] iterate(::POMDPSimulators.POMDPSimIterator{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),VDPTagPOMDP{Nothing},POMCPPlanner{VDPTagPOMDP{Nothing},BasicPOMCP.SolvedPORollout{RandomPolicy{Random._GLOBAL_RNG,VDPTagPOMDP{Nothing},BeliefUpdaters.NothingUpdater},BeliefUpdaters.NothingUpdater,Random._GLOBAL_RNG},Random._GLOBAL_RNG},ParticleFilters.BasicParticleFilter{VDPTagPOMDP{Nothing},VDPTagPOMDP{Nothing},ParticleFilters.LowVarianceResampler,Random._GLOBAL_RNG,Array{TagState,1}},Random._GLOBAL_RNG,ParticleFilters.ParticleCollection{TagState},TagState}, ::Tuple{Int64,TagState,ParticleFilters.ParticleCollection{TagState}}) at /home/fengliu/.julia/packages/POMDPSimulators/4scij/src/stepthrough.jl:92
[8] iterate at /home/fengliu/.julia/packages/POMDPSimulators/4scij/src/stepthrough.jl:86 [inlined]
[9] _collect(::UnitRange{Int64}, ::POMDPSimulators.POMDPSimIterator{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),VDPTagPOMDP{Nothing},POMCPPlanner{VDPTagPOMDP{Nothing},BasicPOMCP.SolvedPORollout{RandomPolicy{Random._GLOBAL_RNG,VDPTagPOMDP{Nothing},BeliefUpdaters.NothingUpdater},BeliefUpdaters.NothingUpdater,Random._GLOBAL_RNG},Random._GLOBAL_RNG},ParticleFilters.BasicParticleFilter{VDPTagPOMDP{Nothing},VDPTagPOMDP{Nothing},ParticleFilters.LowVarianceResampler,Random._GLOBAL_RNG,Array{TagState,1}},Random._GLOBAL_RNG,ParticleFilters.ParticleCollection{TagState},TagState}, ::Base.HasEltype, ::Base.SizeUnknown) at ./array.jl:571
[10] collect at ./array.jl:560 [inlined]
[11] collect_history(::POMDPSimulators.POMDPSimIterator{(:s, :a, :sp, :o, :r, :t, :action_info, :b, :bp, :update_info),VDPTagPOMDP{Nothing},POMCPPlanner{VDPTagPOMDP{Nothing},BasicPOMCP.SolvedPORollout{RandomPolicy{Random._GLOBAL_RNG,VDPTagPOMDP{Nothing},BeliefUpdaters.NothingUpdater},BeliefUpdaters.NothingUpdater,Random._GLOBAL_RNG},Random._GLOBAL_RNG},ParticleFilters.BasicParticleFilter{VDPTagPOMDP{Nothing},VDPTagPOMDP{Nothing},ParticleFilters.LowVarianceResampler,Random._GLOBAL_RNG,Array{TagState,1}},Random._GLOBAL_RNG,ParticleFilters.ParticleCollection{TagState},TagState}, ::Val{false}, ::Nothing) at /home/fengliu/.julia/packages/POMDPSimulators/4scij/src/history_recorder.jl:185
[12] simulate(::HistoryRecorder, ::VDPTagPOMDP{Nothing}, ::POMCPPlanner{VDPTagPOMDP{Nothing},BasicPOMCP.SolvedPORollout{RandomPolicy{Random._GLOBAL_RNG,VDPTagPOMDP{Nothing},BeliefUpdaters.NothingUpdater},BeliefUpdaters.NothingUpdater,Random._GLOBAL_RNG},Random._GLOBAL_RNG}, ::ParticleFilters.BasicParticleFilter{VDPTagPOMDP{Nothing},VDPTagPOMDP{Nothing},ParticleFilters.LowVarianceResampler,Random._GLOBAL_RNG,Array{TagState,1}}, ::VDPTag2.VDPInitDist, ::TagState) at /home/fengliu/.julia/packages/POMDPSimulators/4scij/src/history_recorder.jl:109
[13] simulate(::HistoryRecorder, ::VDPTagPOMDP{Nothing}, ::POMCPPlanner{VDPTagPOMDP{Nothing},BasicPOMCP.SolvedPORollout{RandomPolicy{Random._GLOBAL_RNG,VDPTagPOMDP{Nothing},BeliefUpdaters.NothingUpdater},BeliefUpdaters.NothingUpdater,Random._GLOBAL_RNG},Random._GLOBAL_RNG}, ::ParticleFilters.BasicParticleFilter{VDPTagPOMDP{Nothing},VDPTagPOMDP{Nothing},ParticleFilters.LowVarianceResampler,Random._GLOBAL_RNG,Array{TagState,1}}, ::VDPTag2.VDPInitDist) at /home/fengliu/.julia/packages/POMDPSimulators/4scij/src/history_recorder.jl:85
[14] simulate at /home/fengliu/.julia/packages/POMDPSimulators/4scij/src/history_recorder.jl:59 [inlined]
[15] simulate(::HistoryRecorder, ::VDPTagPOMDP{Nothing}, ::POMCPPlanner{VDPTagPOMDP{Nothing},BasicPOMCP.SolvedPORollout{RandomPolicy{Random._GLOBAL_RNG,VDPTagPOMDP{Nothing},BeliefUpdaters.NothingUpdater},BeliefUpdaters.NothingUpdater,Random._GLOBAL_RNG},Random._GLOBAL_RNG}) at /home/fengliu/.julia/packages/POMDPSimulators/4scij/src/history_recorder.jl:58
[16] top-level scope at /home/fengliu/code/juliaPOMDP/test.jl:13
[17] include at ./boot.jl:328 [inlined]
[18] include_relative(::Module, ::String) at ./loading.jl:1105
[19] include(::Module, ::String) at ./Base.jl:31
[20] exec_options(::Base.JLOptions) at ./client.jl:287
[21] _start() at ./client.jl:460
in expression starting at /home/fengliu/code/juliaPOMDP/test.jl:13

UndefVarError: DDNNode not defined

Hey Zach, I am getting this error: UndefVarError: DDNNode not defined after updating some of my dependencies for VDPTag. Any suggestions on resolving this? I could not find where DDNNode was defined.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.