exanauts / exaadmm.jl Goto Github PK

View Code? Open in Web Editor NEW

16.0 16.0 2.0 1.18 MB

Julia implementation of ADMM solver on multiple GPUs

License: MIT License

Julia 100.00%

admm gpu-computing julia

exaadmm.jl's People

Contributors

Stargazers

Watchers

Forkers

wzhangw

exaadmm.jl's Issues

Need detection of infeasibility

Check out eq.(3.8) in https://link.springer.com/article/10.1007/s00186-020-00713-x

Incorrect initialization of single-period ACOPF

@wzhangw found the initialization of single-period ACOPF on GPU was not correct or compatible to that for CPU.

ACOPF rolling horizon not working

There may be some bug from refactoring ModelAcopf.

ERROR: type ModelAcopf has no field ngen
Stacktrace:
 [1] getproperty
   @ ./Base.jl:42 [inlined]
 [2] admm_restart_rolling(env::ExaAdmm.AdmmEnv{Float64, CuArray{Float64, 1}, CuArray{Int64, 1}, CuArray{Float64, 2}}, mod::ExaAdmm.ModelAcopf{Float64, CuArray{Float64, 1}, CuArray{Int64, 1}, CuArray{Float64, 2}}; start_period::Int64, end_period::Int64, result_file::String)
   @ ExaAdmm ~/.julia/packages/ExaAdmm/srowl/src/models/acopf/acopf_admm_rolling_gpu.jl:24
 [3] solve_acopf_rolling(case::String, load_prefix::String; case_format::String, outer_iterlim::Int64, inner_iterlim::Int64, rho_pq::Float64, rho_va::Float64, obj_scale::Float64, scale::Float64, use_gpu::Bool, use_linelimit::Bool, use_projection::Bool, tight_factor::Float64, outer_eps::Float64, gpu_no::Int64, verbose::Int64, ramp_ratio::Float64, start_period::Int64, end_period::Int64, result_file::String)
   @ ExaAdmm ~/.julia/packages/ExaAdmm/srowl/src/interface/solve_acopf_rolling.jl:28

Parallelize multiple time periods in ucmp/mpacopf

    # TODO: Parallelize the time horizon. Can we parallelize this?

ExaAdmm.jl/src/models/ucmp/ucmp_admm_update_x_gpu.jl

Line 15 in 8638503

for i=1:mpmod.len_horizon

Residual balancing scheme

This is simple to implement based on https://arxiv.org/pdf/1704.06209.pdf. It looks effective for QPsub.

This requires #55.

Parser

Can we rely on the PowerModels.jl parser to avoid ambiguity between the parsed case files.

This work was done by @frapac in exanauts/ExaTron.jl#35 for ExaTron.jl. I guess it would be similar here. I can take a stab.

Integrate KernelAbstractions.jl

To run on AMD and Intel GPUs we need to use a portability layer like KernelAbstractions.jl.

The goal is to add KA to the already existent CPU and CUDA.jl code without changing those.

Abstract away the Array type (Array, CuArray) and use multiple dispatch with a Device type or Array type (not use_gpu)
Add KA kernels
Integrate KA kernels from ExaTronKernels.jl into ExaTron.jl and release as ExaTron 2.0.

Comments and discussions are welcome! @kibaekkim @youngdae @frapac

Return Vm and Va in Solution object

Right now, Solution does not contain Vm and Va. This required in exanauts/ProxAL.jl#72 .

ExaAdmm.jl/src/utils/environment.jl

Line 175 in 18dcaee

mutable struct Solution{T,TD} <: AbstractSolution{T,TD}

Two separate CUDA kernels in ucmp?

Why should this be separate from the above? Why does this have to use a different CUDA setting?

ExaAdmm.jl/src/models/ucmp/ucmp_admm_update_x_gpu.jl

Lines 42 to 47 in 8638503

 CUDA.@sync @cuda threads=64 blocks=(div(ngen-1,64)+1) ucmp_update_uc_membuf_with_ramping_kernel( 

 i, subdata.ngen, 

 v_ramp.u_curr, v_ramp.z_curr, 

 v_ramp.l_curr, v_ramp.rho, 

 mod.uc_membuf 

 )

Test failure for multiperiod ACOPF

Some tests got failed only on GPU.

Test Summary:                                             | Pass  Fail  Total
Testing ADMM algorithms on GPUs                           |  198    27    225
  Testing [x,xbar,z,l,lz] updates and a solve for ACOPF   |   57           57
  Testing [x,xbar,z,l,lz] updates and a solve for MPACOPF |  141    27    168
    Testing U_GEN t=1                                     |    3            3
    Testing U_GEN t=2                                     |    3            3
    Testing U_GEN t=3                                     |    3            3
    Testing U_BR t=1                                      |    9            9
    Testing U_BR t=2                                      |    9            9
    Testing U_BR t=3                                      |    9            9
    Testing V_GEN t=1                                     |    3            3
    Testing V_GEN t=2                                     |    3            3
    Testing V_GEN t=3                                     |    3            3
    Testing V_BR t=1                                      |    9            9
    Testing V_BR t=2                                      |    9            9
    Testing V_BR t=3                                      |    9            9
    Testing Z_GEN t=1                                     |    3            3
    Testing Z_GEN t=2                                     |    3            3
    Testing Z_GEN t=3                                     |    3            3
    Testing Z_BR t=1                                      |    9            9
    Testing Z_BR t=2                                      |    9            9
    Testing Z_BR t=3                                      |    9            9
    Testing L_GEN t=1                                     |    3            3
    Testing L_GEN t=2                                     |    3            3
    Testing L_GEN t=3                                     |    3            3
    Testing L_BR t=1                                      |          9      9
    Testing L_BR t=2                                      |          9      9
    Testing L_BR t=3                                      |          9      9
ERROR: LoadError: Some tests did not pass: 198 passed, 27 failed, 0 errored, 0 broken.

sync/wait in SQOPF.jl test

Issue with #43: observe sync/wait in SQOPF test with ExaAdmm branch https://github.com/exanauts/ExaAdmm.jl/tree/bowen/qp

testing with ] test, SQOPF stuck with sync/wait error as
error.txt
Running the following code block, SQOPF returns normally.

using SQP
using LinearAlgebra
using CUDA

case = "test/algorithms/case9.m"

#SQP with QP solver: ExaADMM with GPU
sqp1 = SQOPF.sqp_solve(case; itermax = 30, eps_inf = 1e-4, eps_res = 1e-4, eps_dir = 1e-4, pen = 5e5, TR = 10, lambda = 0.0, output_flag = 1, outer_iterlim=5000, rho_pq=5000.0, rho_va=5000.0, obj_scale=1.0, scale=1e-4, outer_eps=1e-4, verbose = 0, use_ipopt = false, use_feas_check = false, use_gpu = true, gpu_no = 1);

Use `GridData` structure defined in `utils/grid_data.jl`

Complete the refactoring

These are TODOs for refactoring.

Refactor the mpec model
Refactor the pf model
#15
#13

Remove `use_twolevel::AdmmEnv`

Variable use_twolevel::AdmmEnv is not used in the code. We can remove the variable and clean up the code as if use_twolevel=false. As a result, SolutionTwoLevel can be removed, and SolutionOneLevel may better be renamed to a generic one.

Check dependency for release

We need to remove deps directory and update Project.toml. This requires the new release (v1.0.0) of ExaTron.jl: exanauts/ExaTron.jl#40 (review)

Renaming `AbstractOPFModel`?

It may not need specific to OPF.

ExaAdmm.jl/src/common/environment.jl

Line 360 in 350fa51

abstract type AbstractOPFModel{T,TD,TI,TM} end

Maybe rename it to AbstractModel?

Remove unused folders

As part of refactoring #6 we need to remove unused folders.

Data-specific member in `AdmmEnv`

ExaAdmm.jl/src/common/environment.jl

Line 90 in 34a25e8

load::Load{TM}

This looks like a data-specific member in AdmmEnv struct.

TagBot trigger issue

This issue is used to trigger TagBot; feel free to unsubscribe.

If you haven't already, you should update your TagBot.yml to include issue comment triggers.
Please see this post on Discourse for instructions and more details.

If you'd like for me to do this for you, comment TagBot fix on this issue.
I'll open a PR within a few hours, please be patient!

Normalized residuals and absolute/relative tolerances

We can generalize the tolerance-based termination criteria according to this paper: https://arxiv.org/pdf/1704.06209.pdf. The current implementation terminates ADMM based on the absolute tolerance only.

Timings

@youngdae I see a lot of CUDA.@timed @cuda calls, like here

ExaAdmm.jl/src/models/acopf/acopf_admm_update_x_gpu.jl

Line 26 in e62f009

 time_br = CUDA.@timed @cuda threads=32 blocks=data.nline shmem=shmem_size auglag_linelimit_two_level_alternative( 

This only times the kernel launch or? Should we replace those with CUDA.@timed CUDA.@sync @cuda or does that lead to a slowdown?

Multiperiod ACOPF on GPU

On branch `youngdae/abstraction`

Multiperiod ACOPF (mpacopf) is not passing the test on gpu.

`[email protected]`

mpacopf runs the test only with [email protected]. Even this fails some tests for the values of L_BR.

`[email protected]` and later

The test returns the following error:

Testing [x,xbar,z,l,lz] updates and a solve for MPACOPF: Error During Test at /home/kimk/REPOS/ExaAdmm.jl/test/algorithms/mpacopf_update_gpu.jl:1
  Got exception outside of a @test
  GPU compilation of kernel auglag_generator_kernel(Int64, Int64, Int64, Int64, Int64, Float64, Float64, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceMatrix{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, CuDeviceVector{Float64, 1}, Float64) failed
  KernelError: kernel returns a value of type `Union{}`

  Make sure your kernel function ends in `return`, `return nothing` or `nothing`.
  If the returned value is of type `Union{}`, your Julia code probably throws an exception.
  Inspect the code with `@device_code_warntype` for more details.

Separating data and environment

ExaAdmm.jl/src/common/environment.jl

Lines 85 to 97 in 350fa51

 mutable struct AdmmEnv{T,TD,TI,TM} <: AbstractAdmmEnv{T,TD,TI,TM} 

 case::String 

 data::OPFData 

 load::Load{TM} 

 initial_rho_pq::Float64 

 initial_rho_va::Float64 

 tight_factor::Float64 

 horizon_length::Int 

 use_gpu::Bool 

 use_linelimit::Bool 

 use_twolevel::Bool 

 use_mpi::Bool 

 load_specified::Bool

Some fields are relevant to data. These fields may better be separated out from AdmmEnv struct.

Release

The first version will be released after the abstraction going on in branch youngdae/abstraction.

	CUDA.@sync @cuda threads=64 blocks=(div(ngen-1,64)+1) ucmp_update_uc_membuf_with_ramping_kernel(
	i, subdata.ngen,
	v_ramp.u_curr, v_ramp.z_curr,
	v_ramp.l_curr, v_ramp.rho,
	mod.uc_membuf
	)

	mutable struct AdmmEnv{T,TD,TI,TM} <: AbstractAdmmEnv{T,TD,TI,TM}
	case::String
	data::OPFData
	load::Load{TM}
	initial_rho_pq::Float64
	initial_rho_va::Float64
	tight_factor::Float64
	horizon_length::Int
	use_gpu::Bool
	use_linelimit::Bool
	use_twolevel::Bool
	use_mpi::Bool
	load_specified::Bool

exanauts / exaadmm.jl Goto Github PK

exaadmm.jl's People

Contributors

Stargazers

Watchers

Forkers

exaadmm.jl's Issues

On branch youngdae/abstraction

[email protected] and later

Recommend Projects

Recommend Topics

Recommend Org

On branch `youngdae/abstraction`

`[email protected]` and later