Flappy Bird with Q-Learning implementation on Xilinx Digilent Basys 3 Artix-7 FPGA Board.

Flappy Bird is a 2D mobile game. A bird named “Faby” continuously move to the left. The bird briefly flaps (jump) upward each time we pressed the space key. A set of pipes keep moving right. The bird died once it touch the pipes, ground or sky. We implement the game on the FPGA board. Using keyboard as input and VGA as output. Also, we use Q-Learning method to train the bird flaps by it self on the same FPGA board. Once it’s fully trained, it should be able to set the high score.

• Xilinx Digilent Basys 3 Artix-7 FPGA Board.
• Vivado Design Suite - HLx Editions
• A monitor and a VGA cable.
• A keyboard.

• From source

  • Main code: src/scouce/core/
  • Ip(block memory and keyboard control): src/source/
  • Constrain file: src/source/constrain
  • Original images: src/img/img/
  • coe files converted from original images: src/img/coe

• Open porject directly

  • flappyQ2/flappyQ2.xpr

• VGA: Connect to monitor as output.
• USB: Connect to keyboard. Space key is the input.
• SW0: Switch mode between keyboard control and Q-Learning mode.
• Keyboard mode: Switch SW0 to 0 (down position). Use space key on the keyboard to control the bird.
• Q-Learning mode: Switch SW0 to 1 (up position). Only switch to Q-Learning mode when the game is at HOLD state. That is, the bird stay at original position and no pipes appear on the screen.

• Basic size info (in pixels)
  • Screen: 512 * 288
  • pipe width: 52
  • bird: 23 * 34
• get_bird_pos
• get_pipes_pos
• get_background_offset

• pixel_gen
• vga_controller

• qlearning
• getQLState

• collision_detection
• KeyboardDecoder
• Clock_Divider

• Qtable

𝑆𝑥 = 𝑥/𝑅𝑒𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛, 𝑆𝑦 = 𝑦/𝑅𝑒𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛, 𝑅𝑒𝑠𝑜𝑙𝑢𝑡𝑖𝑜𝑛 = 20

• Algorithm

Initialize Qtable
repeat
    if (is_collide)
        Q[pSx][pSy][pA] = (1-alpha)*Q[pSx][pSy][pA] + alpha*(reward_dead + gamma * max(Q[Sx][Sy]))
    else
        Q[pSx][pSy][pA] = (1-alpha)*Q[pSx][pSy][pA] + alpha*(reward_alive + gamma * max(Q[Sx][Sy]))
        if (Q[Sx][Sy][0] >= Q[Sx][Sy][1])
            don't jump
        else
            jump

SarvagyaVaish's project enhuiz's project