Switchdin challenge for Python Software Engineer (Embedded, Data Science)

Environment setup:

• MQTT message broker: RabbitMQ
• Publish/Subscribe library: paho-mqtt

System overview:

Initialise RabbitMQ server:

# Starting server
rabbitmq-server

# Enabling plugin
rabbitmq-plugins enable rabbitmq_mqtt

# Setup users and authentication
rabbitmqctl add_user mqtt-test mqtt-test
rabbitmqctl set_permissions -p / mqtt-test ".*" ".*" ".*"
rabbitmqctl set_user_tags mqtt-test management

Generate hidden environment file (secret keys)

• File name: .env
• File content:

MQTT-USERNAME = "mqtt-test"
MQTT-PASSWORD = "mqtt-test"

Configure host:

• Modify host in config.json to "localhost".

Prerequisite: Installation Docker on Linux (for AWS EC2) -> read here

docker pull rabbitmq
sudo docker run -d -p 1883:1883 rabbitmq:latest
sudo docker container ps
# grab container name
sudo docker exec -it <container_name> sh

# <!-- Inside container --!>
rabbitmq-plugins enable rabbitmq_mqtt
rabbitmqctl add_user mqtt-test mqtt-test
rabbitmqctl set_permissions -p / mqtt-test ".*" ".*" ".*"
rabbitmqctl set_user_tags mqtt-test management
exit
# <!-- End container --!>

Configure host:

• Modify host in config.json to AWS EC2's public IP address.

Open port of AWS EC2:

• Security group > Inbound rules > IPv4 > "All traffic" (all protocol, all port range, source=0.0.0.0/0) (for demo only, not a good practice to open all the ports and inbound)

Generates a random number between [1 and 100] every [1 to 30] seconds and publishes to broker with topic named app1/randint.

Run app:

cd src
python ./app1.py

Subscribes to topic app1/randint and compute average every [1, 5, 30] minutes, and publishes accordingly to broker with topics named app2/stat_1m, app2/stat_5m, app2/stat_30m.

Run app:

cd src
python ./app2.py

Subscribes to topics app2/stat_1m, app2/stat_5m, app2/stat_30m, and prints out tabular view in console accordingly.

Run app:

cd src
python ./app3.py

• Self-study & research: 1.5 day
• Implementation: 1 day
  • Design: 30 minutes
  • Coding: 2 hours
  • Testing (local): 1 hour
  • Deployment: 2 hour
  • Testing (deploy): 1 hour
  • Documentation: 1 hour

1. RabbitMQ:
   • cli guide
   • MQTT Plugin
   • Tutorials
2. Paho MQTT:
   • Installation and usage
   • Documentations
   • Tutorials
3. Install docker on Linux:
   • Installation guide