Git Product home page Git Product logo

nyaa-1's Introduction

NyaaV2

Setup

Setting up MySQL/MariaDB database for advanced functionality

  • Enable USE_MYSQL flag in config.py
  • Install latest mariadb by following instructions here https://downloads.mariadb.org/mariadb/repositories/
    • Tested versions: mysql Ver 15.1 Distrib 10.0.30-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2
  • Run the following commands logged in as your root db user:
    • CREATE USER 'test'@'localhost' IDENTIFIED BY 'test123';
    • GRANT ALL PRIVILEGES ON * . * TO 'test'@'localhost';
    • FLUSH PRIVILEGES;
    • CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
  • To setup and import nyaa_maria_vx.sql:
    • mysql -u <user> -p nyaav2
    • DROP DATABASE nyaav2;
    • CREATE DATABASE nyaav2 DEFAULT CHARACTER SET utf8 COLLATE utf8_bin;
    • SOURCE ~/path/to/database/nyaa_maria_vx.sql

Finishing up

  • Run python db_create.py to create the database
  • Load the .sql file
    • mysql -u user -p nyaav2
    • SOURCE cocks.sql
    • Remember to change the default user password to an empty string to disable logging in
  • Start the dev server with python run.py
  • When you are finished developing, deactivate your virtualenv with source deactivate

Enabling ElasticSearch

Basics

Enable MySQL Binlogging

  • Add the [mariadb] bin-log section to my.cnf and reload mysql server
  • Connect to mysql
  • SHOW VARIABLES LIKE 'binlog_format';
    • Make sure it shows ROW
  • Connect to root user
  • GRANT REPLICATION SLAVE ON *.* TO 'test'@'localhost'; where test is the user you will be running sync_es.py with

Setting up ES

  • Run ./create_es.sh and this creates two indicies: nyaa and sukebei
  • The output should show acknowledged: true twice
  • The safest bet is to disable the webapp here to ensure there's no database writes
  • Run python import_to_es.py with SITE_FLAVOR set to nyaa
  • Run python import_to_es.py with SITE_FLAVOR set to sukebei
  • These will take some time to run as it's indexing

Setting up sync_es.py

  • Sync_es.py keeps the ElasticSearch index updated by reading the BinLog
  • Configure the MySQL options with the user where you granted the REPLICATION permissions
  • Connect to MySQL, run SHOW MASTER STATUS;.
  • Copy the output to /var/lib/sync_es_position.json with the contents {"log_file": "FILE", "log_pos": POSITION} and replace FILENAME with File (something like master1-bin.000002) in the SQL output and POSITION (something like 892528513) with Position
  • Set up sync_es.py as a service and run it, preferably as the system/root
  • Make sure sync_es.py runs within venv with the right dependencies

Enable the USE_ELASTIC_SEARCH flag in config.py, restart the application, and you're good to go.

Database migrations

  • Uses flask-Migrate
  • Run ./db_migrate.py db migrate to generate the migration script after database model changes.
  • Take a look at the result in migrations/versions/... to make sure nothing went wrong.
  • Run ./db_migrate.py db upgrade to upgrade your database.

Code Quality:

  • Remember to follow PEP8 style guidelines and run ./lint.sh before committing.

nyaa-1's People

Contributors

theamm avatar aldacron avatar nyaadev avatar sharkykh avatar unknown-consortium avatar reimuhakurei avatar snowfag avatar qqueue avatar ricardo1991 avatar katnyaa avatar nathancyam avatar martstern avatar nazonaze avatar sn0wcrack avatar kyamiko avatar erengy avatar wranai avatar jsfernandes avatar kevinji avatar counterpillow avatar fallenwarrior2k avatar bittebitte avatar tipuch avatar

Watchers

James Cloos avatar Kurumi avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.