Git Product home page Git Product logo

android-analytics's Introduction

Android analytics

Web analytics software so simple it runs on your phone.

This is a damn fine cup of coffee.

"This is a damn fine cup of coffee." -- Agent Cooper

Table of Contents

About

Screenshot of the app running locally, a few charts and a map are shown

Here's an in-depth blog post about this project.

This is a web analytics tool that runs on Android devices. If you're just curious about who visits your personal page, this might be good enough for you.

It's really just a glorified nginx log visualization tool. Annonymized visits are stored in your phone, put into a database, and viewed with Grafana.

How this works

diagram

When someone visits your website, Javascript sends a GET <ngrok-domain>/damn_fine_coffee[1]. Ngrok will redirect that to your phone, in which Nginx will create a log entry (it doesn't log other endpoints[1]).

Every day a cron job processes the day's logs[2]. Invalid logs (duplicated requests or requests originating from non-monitored domains) are discarded, IPs are annonymized using MD5 hashing, and geo info (city-level) is calculated from the IPs. Each valid log entry becomes a row of the hits table.

The hits table is then used by Grafana to create all kinds of visualization -- at this point its up to you!

Requirements

  1. An Android phone running Termux (recommended: set up SSH remote access). Remember to press "Acquire wakelock" on Termux as well;
  2. An ngrok account (or any equivalent service)

Installation

In your Android phone:

  1. Clone this repository and cd into it
  2. Create your .env config file with cp -n .env.template .env and add your passwords, domains etc. If you don't have a SSL certificate, comment out the SSL-related parts of config/nginx.conf (more on that in the About SSL section below)
  3. Run the setup script: bash bin/setup.sh
  4. Install ngrok on your phone (search for "install ngrok on termux" for a bunch of tutorials showing how)
  5. Run ngrok with ngrok http localhost:8080, take note of the URL
  6. In the website you want to monitor, add a call to your ngrok URL: <script> fetch("https://my-ngrok-domain/damn_fine_coffee"); </script>
  7. Install Grafana on Termux (I used AnLinux for this)

All set! Run tail -f log/nginx.access.log on your phone and try visiting your monitored website. You should see the request getting logged.

Next step is to manually run the compiler job (./bin/compile_logs.sh), which will send that request to the DB (by default it is ran daily at 23:59 -- just run it manually once to see if everything is OK).

Open http://<your-phones-local-ip>:3000 to open Grafana and create some charts! ๐ŸŽ‰

Notes

[1] The web is full of crawlers, mostly hitting /, but also /tp-link, /admin etc. We want to just ignore those.

[2] Leveraging Nginx log rotation.

android-analytics's People

Contributors

lbrito1 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.