Git Product home page Git Product logo

alexandria's Introduction

Alexandria DHT

Join the chat at https://gitter.im/ethereum/alexandria Build Status PyPI version Python versions Docs build

Client for the Alexandria DHT network

Read more in the documentation on ReadTheDocs. View the change log.

Quickstart

pip install alexandria

Developer Setup

If you would like to hack on alexandria, please check out the Snake Charmers Tactical Manual for information on how we do:

  • Testing
  • Pull Requests
  • Code Style
  • Documentation

Development Environment Setup

You can set up your dev environment with:

git clone [email protected]:ethereum/alexandria.git
cd alexandria
virtualenv -p python3 venv
. venv/bin/activate
pip install -e .[dev]

Testing Setup

During development, you might like to have tests run on every file save.

Show flake8 errors on file change:

# Test flake8
when-changed -v -s -r -1 alexandria/ tests/ -c "clear; flake8 alexandria tests && echo 'flake8 success' || echo 'error'"

Run multi-process tests in one command, but without color:

# in the project root:
pytest --numprocesses=4 --looponfail --maxfail=1
# the same thing, succinctly:
pytest -n 4 -f --maxfail=1

Run in one thread, with color and desktop notifications:

cd venv
ptw --onfail "notify-send -t 5000 'Test failure ⚠⚠⚠⚠⚠' 'python 3 test on alexandria failed'" ../tests ../alexandria

Release setup

For Debian-like systems:

apt install pandoc

To release a new version:

make release bump=$$VERSION_PART_TO_BUMP$$

How to bumpversion

The version format for this repo is {major}.{minor}.{patch} for stable, and {major}.{minor}.{patch}-{stage}.{devnum} for unstable (stage can be alpha or beta).

To issue the next version in line, specify which part to bump, like make release bump=minor or make release bump=devnum. This is typically done from the master branch, except when releasing a beta (in which case the beta is released from master, and the previous stable branch is released from said branch).

If you are in a beta version, make release bump=stage will switch to a stable.

To issue an unstable version when the current version is stable, specify the new version explicitly, like make release bump="--new-version 4.0.0-alpha.1 devnum"

alexandria's People

Contributors

carver avatar cburgdorf avatar davesque avatar njgheorghita avatar pipermerriam avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

alexandria's Issues

CacheDB concurrency issue

Traceback (most recent call last):
  File "/usr/local/bin/alexandria", line 11, in <module>
    load_entry_point('alexandria', 'console_scripts', 'alexandria')()
  File "/root/alexandria/alexandria/_boot.py", line 53, in _boot
    trio.run(manager.run)
  File "/usr/local/lib/python3.7/dist-packages/trio/_core/_run.py", line 1804, in run
    raise runner.main_task_outcome.error
  File "/usr/local/lib/python3.7/dist-packages/async_service/trio.py", line 208, in run
    for _, exc_value, exc_tb in self._errors
  File "/usr/local/lib/python3.7/dist-packages/async_service/base.py", line 301, in _run_and_manage_task
    await task.run()
  File "/usr/local/lib/python3.7/dist-packages/async_service/trio.py", line 76, in run
    await self._async_fn(*self._async_fn_args)
  File "/root/alexandria/alexandria/_boot.py", line 23, in run
    await main()
  File "/root/alexandria/alexandria/main.py", line 168, in main
    await manager.wait_finished()
  File "/usr/local/lib/python3.7/dist-packages/async_generator/_util.py", line 42, in __aexit__
    await self._agen.asend(None)
  File "/usr/local/lib/python3.7/dist-packages/async_service/trio.py", line 409, in background_trio_service
    await manager.stop()
  File "/usr/local/lib/python3.7/dist-packages/trio/_core/_run.py", line 730, in __aexit__
    raise combined_error_from_nursery
  File "/usr/local/lib/python3.7/dist-packages/async_service/trio.py", line 208, in run
    for _, exc_value, exc_tb in self._errors
  File "/usr/local/lib/python3.7/dist-packages/async_service/base.py", line 301, in _run_and_manage_task
    await task.run()
  File "/usr/local/lib/python3.7/dist-packages/async_service/base.py", line 166, in run
    await self.child_manager.run()
  File "/usr/local/lib/python3.7/dist-packages/async_service/trio.py", line 208, in run
    for _, exc_value, exc_tb in self._errors
  File "/usr/local/lib/python3.7/dist-packages/async_service/base.py", line 301, in _run_and_manage_task
    await task.run()
  File "/usr/local/lib/python3.7/dist-packages/async_service/base.py", line 166, in run
    await self.child_manager.run()
  File "/usr/local/lib/python3.7/dist-packages/async_service/trio.py", line 208, in run
    for _, exc_value, exc_tb in self._errors
  File "/usr/local/lib/python3.7/dist-packages/async_service/base.py", line 301, in _run_and_manage_task
    await task.run()
  File "/usr/local/lib/python3.7/dist-packages/async_service/trio.py", line 76, in run
    await self._async_fn(*self._async_fn_args)
  File "/root/alexandria/alexandria/kademlia.py", line 259, in _handle_content_ingestion
    bundle,
  File "/usr/local/lib/python3.7/dist-packages/trio/_threads.py", line 310, in to_thread_run_sync
    return await trio.hazmat.wait_task_rescheduled(abort)
  File "/usr/local/lib/python3.7/dist-packages/trio/_core/_traps.py", line 165, in wait_task_rescheduled
    return (await _async_yield(WaitTaskRescheduled(abort_func))).unwrap()
  File "/usr/local/lib/python3.7/dist-packages/outcome/_sync.py", line 111, in unwrap
    raise captured_error
  File "/usr/local/lib/python3.7/dist-packages/trio/_threads.py", line 262, in do_release_then_return_result
    return result.unwrap()
  File "/usr/local/lib/python3.7/dist-packages/outcome/_sync.py", line 111, in unwrap
    raise captured_error
  File "/root/alexandria/alexandria/content_manager.py", line 485, in ingest_content
    self._ingest_content_data(content.key, content.data)
  File "/root/alexandria/alexandria/content_manager.py", line 505, in _ingest_content_data
    self.cache_db.set(Content(key, data))
  File "/root/alexandria/alexandria/content_manager.py", line 259, in set
    previous_content = self._get_record(content.key)
  File "/root/alexandria/alexandria/content_manager.py", line 237, in _get_record
    for content in self._records:
RuntimeError: deque mutated during iteration

Probably because it's using threads for content ingestion...

Detect when advertisement is bottleneck

Currently advertisement of content attempts to broadcast content once ever KADEMLIA_ANNOUNCE_INTERVAL. With a big enough content database, the advertisement queue will lag behind. We should detect this and report it as a warning condition.

More intelligent periodic announcements

Currently the nodes re-advertise all of their content every 30 minutes. This isn't viable.

Advertisements should be changed to allow multiple advertisements to be grouped into a single message.

The periodic announcement should operate on a policy of announcing the content that was last announced the longest time ago and only announcing content that hasn't been announced since some MIN_REANNOUNCE_TIME constant.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.