• not very fast start of project
• read the link in the assignment
• confusion on the assignment compile-time versus run-time self-evolution.
• Question: most coding will be done is Superapp? Yes ❗
• Upcoming sprint: repair the run-time APK installing
• lacks 1 feature: requires you to be connectable. (ToDo read about: connectable port in Android) or run a known Bittorrent tracker.

from tribler.

What we did this weak:

• We created a test apk that runs inside the app, using the run-time APK installing
• Made/divided tickets on minor improvements on our repo
  • for the tickets see: the repo
• Fixed the run-time APK installing bug. The run-time apk installing now runs on Android 14+
• Set up github repo rules, to ensure a good way of working
• Adds feature that allows renaming of APK

Problems we encountered:

• We encountered the problem where APKs do not torrent
  • We suspect this is an issue with the p2p network
  • We saw sometimes no peers showing up in the debug screen
• Build release apk CI job fails continuously on our github.
  • Is this because of us, or can this just not be fixed.
• Is there an equivalent to pretier in Kotlin? (To format the kotlin files without having to run the gradle thing everytime)

from tribler.

• Possibly a port connectivity issue or not? Test with https://canyouseeme.org/ type of tools
  • Flaky connectivity ? (torrenting works for some team members)
  • Port forwarding in Android studio on Mac just simply works? No need to configure port forwarding between two emulator instances.
  • zero peers found issue
  • debug screen: number of incoming IPv8 UDP packets
• fixed several bugs? Added Android 14 compatibility with safer dynamic code
• To format locally the files ktlint
• Renaming feature: https://github.com/JopSchaap/trustchain-superapp/pull/14/files
• Feel free to use this code from another team
  • #7913 (comment)
  • document the usage please!
  • they got transfers operational and Wireshark-debugging perfected
• Freedom-of-Computing contains a gossiping mechanism which periodically shares local applications to other users and downloads unseen applications from other users from FOC.readme
  • first reproduce the networking debug code of other team
  • add their networking debug to your FOC community
  • other peers you discover
  • add to GUI

Sprint ToDo:

• Hopefully you can make more progress in this course!
• Get wireshark going please
• Understanding of connectivity issues. Connectable at a port
• Study carrier-grade NAT puncturing
• Operational debug screen.

Concrete Week 10 goal:

• FOC is fixed and connectivity issue solved
• FOC of discovery of new code is merged with running code of voting
  • current behaviour is to autodownload anything you find. ✔️ after new code is discovered in gossiper, you automatically download it using Bittorrent. However, installing is not automatic.
  • user decides to delete or install the newly discovered code.
  • this decision is shared with with others in an upgrade of the gossip protocol
  • downvoted and upvoted (no need for multi-sig complexity)
  • collect these votes in background what others voted and show to users as FOC enhancement
• Week 10 test: download your .APK, join network, vote on APK proposals, see votes come in.

from tribler.

Linter for kotlin: https://github.com/pinterest/ktlint

from tribler.

What we did this weak:

• We added a vote button
  • Right now a user is just able to vote once, and cannot cancel their vote
• We added a down-vote button
• We added the ability of the app to store votes by user id
  • We want to make this signed with the user id so peers can share other peers votes
• We restructured the layout of stored app in the file directory, to store apks in a directory which are named by their hash.
  • This is not yet merged since it doesn't work yet for torrented apks.
• We got wirshark going
• We figured out what to do to make the connectivity issue go away
  • We figured out you should have the app installed in two (physically) different devices. (e.g. two emulators on one machine doesn't work)
    • We are however not sure why this is required
• We got the debug screen (from the other group) up and running

Problems we encountered:

Questions:

• Should there be the option to remove votes?
• Does it make sense to get votes for an apk, you do not have torrented yet?
• Is it a problem to gossip all the votes at once, or is there a better approach?

from tribler.

from tribler.

For the next week:

• What is member id? When it is created?
• Display votes on the screen. Now the buttons are not dynamic, find easiest fix for that.
• Restructure vote object: it should have item_id, public key (or member id), signature, freshness, vote strength (-1 down, 0 neutral, 1 up)
• Store what you received from the network, you can always garbage collect later.
• The simplest way to implement the gossip algorithm: send to your neighbours if the message is new. Downside - it might be bad for performance
• A bit better version: push-based with ttl (time to live) inside the vote object. Ttl is an integer (if value is > 0) that indicates if the message should be pushed further. Then you decrement it and push to random k peers you connected to.
• Have data for benchmarking to illustrate how bad/good the gossip is. Simplest way for now record time when message was received, and when was send with the peer_id.

from tribler.

What we did this week

• We restructured the vote objects, to include signatures/ public keys
• Signatures are verified with the associated public keys
• Vote objects now also come in a neutral variant
• We implemented a simple hot potato gossip working, that distributes votes to other peers
• The number of votes are now shown next to the vote up/down button
• A very basic pull based gossip, where newly introduced peers ask for votes they are yet unaware of

Things we ran into this week

• We were unsure how to ensure a peer doesn't generate two keys and uses both to vote twice

Things we want to work on next week

• We want to make it possible to revert your vote
• Improve on sharing votes to new peers (e.g. pull part of the protocol)

from tribler.

Demo of push gossip: https://jam.dev/c/3b808243-706e-4d41-b082-668026a82a53

from tribler.

Next week:

• Fix pull based gossip. For example, you can delay it until we are connected. Other option you can used "extra_bits" for introduction_request/response.

• Get the number from a simple benchmark: Time to convergence (when everyone in the network has discovered the apk), and overhead per vote (bandwidth, ...)

• Now the votes are collected with a queue that is polled every second. What is the easiest way to implement it asynchronously? Implement a singleton? Any ideas @InvictusRMC?

• Check if you can install/reinstall super-app with new apk. (It's a sidequest, so you're on a time budget)

from tribler.

• Now the votes are collected with a queue that is polled every second. What is the easiest way to implement it asynchronously? Implement a singleton? Any ideas @InvictusRMC?

Right now the implementation is indeed very inefficient. What I suggest is making use of a listener. That way we can check whether conditions are met whenever receiving a new block, as opposed to checking periodically.

from tribler.

• Removed all the queues which increased the latency for push & pull-based gossiping
• Improved latency of push based from 5-10 seconds down to 0.1 seconds
• Improved latency of pull based from 10 seconds to 0.2 seconds
• Benchmarked the results and made improvements to reduce message size
• Found that Pull Based gossiping couldn't scale as the message would eventually get too big, so switched to using the EVA protocol and tested that it scales (tested up to 50 votes)
• Fixed a bug that occurred when voting on an APK that was gossiped to us
  
  This graph shows the size of the message as the number of votes increases. The original implementation only scaled to 3 votes, then after reducing the message size it scaled to 4 votes but the latest version that uses the EVA protocol can scale way further. The latency was also improved as previously mentioned, and also removed the operational overhead of repeatedly checking the queue.
  
  The graph shows how we reduced the message size, but we also reduced the latency and removed the operational overhead of repeatedly checking the queue.

from tribler.

• We looked into how installing could work
• There are three solutions we looked into
  1. Replacing the standard apk with a small one that only loads the new apk, and thus would allow new versions
     • Problem1: the current superapp constructs views with xml fragments which cannot be loaded
     • Problem2: One could only install one version
  2. use custom class loader which would allow loading updated versions of classes
     • This would be the most optimal way of loading classes
     • This would allow patching the super-app
     • Problem1: We could not get class loading to work in this way (but we think it is possible, if structurally complecated)
     • Problem2: loading xml files still wouldn't work
  3. We could put installed apps on the home screen:
     • This is what we are currently working on
     • Problem1: loading xml files still wouldn't work

from tribler.

Solid progress

For the next week:

• Keep good documentation to help with onboarding for the next teams
• Exporting to the main screen
  • Priority: to main screen, first manually
  • Then switch to manual import to the main screen
• Gossip improvement: Use short ids when first sending pull-request gossip. The initiator should be the node joining.
• Improve benchmarking figures to make them more interesting: time to convergence and throughput as metrics.
• Side-quest: Learn more about set reconciliation to improve your gossip even more:
  https://github.com/AljoschaMeyer/set-reconciliation
  https://ics.uci.edu/~eppstein/pubs/EppGooUye-SIGCOMM-11.pdf

from tribler.

from tribler.

What we did this week

• We ran tests on the base implementation of pull based vote gossiping see: comment above
• We improved the pull request gossiping, with only sending not yet known votes.
• fixed bugs
• Write kdoc documentation for classes and methods
• Add additional readme for the freedom of computing sub-module see: README.md
• Fix issues with "installing" apps
• add vote threshold installing
• Sync branch with main branch over at tribler

Things we ran into this week

• We were unsure what an appropriate vote treshold would be(right now it is set to 10)
• Should apps also be automatically uninstalled, if # votes drop below treshold?

from tribler.

Improved graph for push-based gossip

APK link
APK Master
Old APK
my-special

from tribler.

• focus on gossip (mostly ignoring the 4G/5G network connectivity issue)
• 500 votes testing
• pull gossip: advertise all votes you have already (ID for a vote; UUID, why not SHA256 (32 bytes) spoofing 💣 ?)
  • infinite growth of advertisement?
  • monthly data bundle gone in 60 seconds?
  • current code: advertise everything you know
• usage of script to generate votes/workload + Android emulator (1x per PC)
• Running code for a democratic process which determines your apps inside superapp
  • each Bittorrent swarm with sufficient votes gets download
  • installed and show on frontpage upon startup 😮
  • Yes, we are aware this lacks security measures !!!
  • Lets focus on this for "Week 10" test
• Nothing scales in this app. Nothing is secure. Will take years/decades to fix!
• Honest conversation: difficult for the app to break outside the tech bubble 🤔
  • why upgrade to this, versus continue to use Tiktok and Youtube ??
  • Needs to go viral and hear friends use it
• Week 10 checklist
  • Running APK
  • possibly the "i" info screen on votes seen
  • live voting test
  • frontpage install test
  • documentation in readme.md
  • release of your .APK ?
  • bonus: make it scale!
  • bonus: create PR of all your improvements upstream

from tribler.

What we did this week:

• We have demo prepared apk prepared at: https://github.com/JopSchaap/trustchain-superapp/releases/tag/demo-release
  • If nothing shows up your preference store might be corrupt because of an old version of the FOCInstaller. To fix you should clear the local storage of the app, see release on how to do that
• We updated the info screen to include Threshold for install value and total number of votes stored
  • Please note that this behavior is not yet merged and thus not shown in the demo

from tribler.

For next week:

• Investigate the bug showing different number of torrents
• Add more apks (10 and more) either into the default app, or prepare a bunch of link with some dummy apks (some sound, image, search)
• During the demo: show upload of apk and gossip of apk.
• Another demo: popular apk auto-install (threshold of three votes)
• Prepare a compilation README file with your decisions, and limitations of your protocols (what works and what is future work)
• Hand in Apk and GitHub repo wit the code.
• @synctext what is the expected deadline to submit the upstream PR?
• Unit tests and good coverage
• Corner cases - broken apk gossip, big transfer. Have tests to show the coverage for corner cases.

from tribler.