kantancoding / microservices-python Goto Github PK

Can you add this Microservice Architecture and System Design with Python & Kubernetes – Full Course to readme

Hi,

I'm following the video on youtube and run successfully all the commands until the third hour when you access the rabbitmq running on localhost.

I cannot access it and my minikube tunnel output looks like:

Status:	
	machine: minikube
	pid: 238475
	route: 10.96.0.0/12 -> 192.168.49.2
	minikube: Running
	services: []
    errors: 
		minikube: no errors
		router: no errors
		loadbalancer emulator: no errors

Connecting to http://rabbitmq-manager.com/ returns "This site can’t be reached".
Any idea why this is happening? I'm new to kubernetes and quite stuck by this issue

Thanks a lot

Andrea

Error Image:

My server file image:

My auth-deployment file image:

My auth-service file image:

My auth-configmap file image:

My Docker file for auth:

My all pods are running successfully:

I don't know what is the error. When id do systemctl status mysql it shows that my mysql is running successfully in my local device.
My operating system is Ubuntu 22.04 LTS.

Please help me to resolve the issue.

You need to enable 2FA on your Google account .

It will allow you to generate an app password for "Mail".

You should be able to send emails now.

Hello,

Many thanks for the really great video. I'm unable to authenticate via the login route. It throws an exception when calling cur = mysql.connection.cursor(). I pinged host.minikube.internal and added that IP to the hosts file. I'm not sure what else I'm supposed to do. Here's what is in my hosts file right now. Am I missing anything?

127.0.0.1 kubernetes.docker.internal # not sure if this is required
192.168.49.1 host.minikube.internal
127.0.0.1 mp3converter.com
127.0.0.1 rabbitmq-manager.com
127.0.0.1 localhost
::1 localhost ip6-localhost ip6-loopback
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters

Traceback (most recent call last):
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/flask/app.py", line 1463, in wsgi_app
response = self.full_dispatch_request()
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/flask/app.py", line 872, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/flask/app.py", line 870, in full_dispatch_request
rv = self.dispatch_request()
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/flask/app.py", line 855, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) # type: ignore[no-any-return]
File "/home/devon/system_design/python/src/auth/server.py", line 27, in login
cur = mysql.connection.cursor()
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/flask_mysqldb/init.py", line 109, in connection
ctx.mysql_db = self.connect
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/flask_mysqldb/init.py", line 97, in connect
return MySQLdb.connect(**kwargs)
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/MySQLdb/init.py", line 121, in Connect
return Connection(*args, **kwargs)
File "/home/devon/system_design/python/src/auth/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 195, in init
super().init(*args, **kwargs2)
MySQLdb.OperationalError: (2003, "Can't connect to MySQL server on 'host.minikube.internal:3306' (111)")

My host's file :

hi, I was going through the tutorial. I created a dockerFile and while building the image of dockerfile. It was unable to install requirement. I removed the particular software name (eg. apturl==0.5.2) from Dockerfile so that the requirement file gets installed but the image was again not built showing same error ( non-zero code: 1) with different software version.
I have atttached my dockerfile and requirement.txt for your [convinience])

Step 3/8 : WORKDIR /app
---> Using cache
---> 9b03bbc62069
Step 4/8 : COPY ./requirements.txt /app
---> Using cache
---> 09b7caf2af3f
Step 5/8 : RUN pip install --no-cache-dir --requirement /app/requirements.txt
requirements.txt

---> Running in 4cf4d2ed059b
ERROR: Could not find a version that satisfies the requirement apturl==0.5.2 (from versions: none)
ERROR: No matching distribution found for apturl==0.5.2
The command '/bin/sh -c pip install --no-cache-dir --requirement /app/requirements.txt' returned a non-zero code: 1

MySQLdb.OperationalError: (2002, "Can't connect to MySQL server on 'host.minikube.internal' (115)")                                              │     43h           │
│ 172.17.0.1 - - [20/Nov/2022 10:19:23] "POST /login HTTP/1.1" 500 -

Anyone has idea how to solve this?

Has this got something to do with /etc/hosts settings?

I could not even open mp3converter.com or rabbitmq-manager.com following the video.
Had to add 192.168.64.2 rabbitmq-manager.com before i could do that, 127.0.0.1 rabbitmq-manager.com didn't work (can anyone explain?)
Not sure where 192.168.64.2 comes from or why i must use that in /etc/hosts

My minikube ssh 'cat /etc/hosts' shows

127.0.0.1       localhost
127.0.1.1       minikube
192.168.64.1    host.minikube.internal
192.168.64.2    control-plane.minikube.internal

if this is helpful (not sure what to make of this output)

How do I run the application?

It would be very helpful if instructions could be added to README.

Hi,

when calling
curl -X POST http://mp3converter.com/login -u andrea@localhost:password

I have this error trying to connect to the MySQL db

  File "/app/server.py", line 25, in login
    cur = mysql.connection.cursor()
  File "/usr/local/lib/python3.10/site-packages/flask_mysqldb/__init__.py", line 101, in connection
    ctx.mysql_db = self.connect
  File "/usr/local/lib/python3.10/site-packages/flask_mysqldb/__init__.py", line 88, in connect
    return MySQLdb.connect(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/__init__.py", line 123, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 185, in __init__
    super().__init__(*args, **kwargs2)
TypeError: 'str' object cannot be interpreted as an integer

My login in server.py:

import jwt, datetime, os
from flask import Flask, request
from flask_mysqldb import MySQL

server = Flask(__name__)
mysql = MySQL(server)


# config
server.config["MYSQL_HOST"] = os.environ.get("MYSQL_HOST")
server.config["MYSQL_USER"] = os.environ.get("MYSQL_USER")
server.config["MYSQL_PASSWORD"] = os.environ.get("MYSQL_PASSWORD")
server.config["MYSQL_DB"] = os.environ.get("MYSQL_DB")
server.config["MYSQL_PORT"] = os.environ.get("MYSQL_PORT")
# print(server.config["MYSQL_HOST"])


@server.route("/login", methods=["POST"])
def login():
    auth = request.authorization
    if not auth:
        return "missing credentials", 401

    # check db for user and pass
    cur = mysql.connection.cursor()
    res = cur.execute(
        "SELECT email, password FROM user WHERE email=%s", (auth.username,)
    )

    if res > 0:
        user_row = cur.fetchone()
        email = user_row[0]
        password = user_row[1]

        if auth.username != email or auth.password != password:
            return "invalid credentials", 401
        else:
            return createJWT(auth.username, os.environ.get("JWT_SECRET"), True)
    else:
        return "invalid credentials", 401

...

auth-configmap.yaml

apiVersion: v1
kind: ConfigMap
metadata:
  name: auth-configmap
data:
  MYSQL_HOST: host.minikube.internal
  MYSQL_USER: auth_user
  MYSQL_DB: auth
  MYSQL_PORT: "3306"

my init.sql:

CREATE USER 'auth_user'@'localhost' IDENTIFIED BY 'Aauth123';

CREATE DATABASE auth;

GRANT ALL PRIVILEGES ON auth.* TO 'auth_user'@'localhost';

USE auth;

CREATE TABLE user (
	id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
	email VARCHAR(255) NOT NULL UNIQUE,
	password VARCHAR(255) NOT NULL
);

INSERT INTO user (email, password) VALUES ('andrea@localhost', 'password');

not sure about the origin of the error, can you help?

@selikapro

First of all, thank you so much for this course.

Any plans to add MongoDB setup instruction for Microservice Architecture and System Design with Python & Kubernetes – Full Course?

Can we use the approach explained in this(https://www.youtube.com/watch?v=MQZ-WXZbU4w) video? If so, what will be the URI value?

mongo_video = PyMongo(server, uri="mongodb://host.minikube.internal:27017/videos")
mongo_mp3 = PyMongo(server, uri="mongodb://host.minikube.internal:27017/mp3s")

Hey,

I just spent a few hours on the YouTube video and tutorial. Great work! However, I need help at this point.

My dev setup:

I clone this repo on my MacOS M1.

Description of the error:

When I try to upload a document, I get an error: internal server error
Here is my request:
curl -X POST -F 'file=@./test.mkv' -H 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJ1c2VybmFtZSI6InNpbW9ucHJ1ZGhvbW1lQGdtYWlsLmNvbSIsImV4cCI6MTcxMDE1ODUyNCwiaWF0IjoxNzEwMDcyMTI0LCJhZG1pbiI6dHJ1ZX0.no2pRm__ILHtMKtsWlg1yTxXS5yJllGtAed9HZJgEyo' http://mp3converter.com/upload

I am able to acquire a token.
In the gateway pod logs in k9s I see that the requests for POST /login -> 200 and POST /upload -> 500.
The minikube tunnel is always open, I have all the services running. The queues are define in RabbitMQ Managment.
Also, I reset the gateway service to try to reconnect it to rabbitmq, as explained in the video.

rabbit/manifests/ingress.yaml:

rabbit/manifests/service.yaml:

gateway/server.p

y:

RabbitMQ Management

Hi there,

I encountered an error when I was trying to post a random video file to the server. However, every time I was trying to POST the file using cURL, I got the following response. I tried modifying the ingress.yaml file but it did not work out. Just wondering if you have any ideas on how to solve the issue. Thank you.

Code tried in Terminal:

curl -X POST -F 'file=@./test.mp4' -H 'Authorization: Bearer TOKEN' http://mp3converter.com/upload

Response from server

<html>
<head><title>413 Request Entity Too Large</title></head>
<body>
<center><h1>413 Request Entity Too Large</h1></center>
<hr><center>nginx</center>
</body>
</html>

Is there any reason for keeping the venv directories in the repo? They are platform-specific and cause significant bloat. See also: #2.

Error image:

My server.py file: I think there must be something wrong with my MONGO_URI, but somehow, cannot see where this setting is came from, even in your tutorial.

My util.py at /storage: Please note that I change the try-except so my server one can print out the error, dunno why I cannot use the print error to see in the logs command of kubectl

Am I missing something here?

It may be a Python 3.11 thing ... but it looks like an error gets thrown which in turns returns a 500 internal server error when calling http://mp3converter.com/login .... I'm assuming its the port here https://github.com/selikapro/microservices-python/blob/4382751315adadbeabd32a69ba9eb8472f37683c/src/auth/server.py#L13. Casting it as an int should solve your problem.

server.config["MYSQL_PORT"] = int(os.environ.get("MYSQL_PORT"))

Auth pod logs:

Receiving this error when trying to run curl -v -X POST http://mp3converter.com/login -u [email protected]:Admin123

* Serving Flask app 'server'
 * Debug mode: off
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:5000
 * Running on http://172.17.0.15:5000
Press CTRL+C to quit
[2022-11-21 02:23:54,882] ERROR in app: Exception on /login [POST]
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 2525, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1822, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1820, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python3.10/site-packages/flask/app.py", line 1796, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/app/server.py", line 25, in login
    cur = mysql.connection.cursor()
  File "/usr/local/lib/python3.10/site-packages/flask_mysqldb/__init__.py", line 101, in connection
    ctx.mysql_db = self.connect
  File "/usr/local/lib/python3.10/site-packages/flask_mysqldb/__init__.py", line 88, in connect
    return MySQLdb.connect(**kwargs)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/__init__.py", line 123, in Connect
    return Connection(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/MySQLdb/connections.py", line 185, in __init__
    super().__init__(*args, **kwargs2)
TypeError: 'str' object cannot be interpreted as an integer
172.17.0.1 - - [21/Nov/2022 02:23:54] "POST /login HTTP/1.1" 500 -

Add guide to the readme file about local run and development.

Add instructions to setup minikube locally and kubernetes

Here is my curl command, not sure why this error

I followed along till writing the Dockerfile and did the pip3 freeze but then when i ran the script, mysqlclient is creating problems and is interrupting the build process. I faced a similar issue during the initial stage too when you once did the localhost print statement but then, everything after that step went smooth until this error. Im using MacBook Air M1 (2020). Requesting your help. Thanks in advance!

Pika is unable to successfully spin up a BlockingConnection.
I am suspecting it is expecting some SSL config, but that was not done in the lessons.

Traceback (most recent call last): File "/app/server.py", line 16, in <module> connection = pika.BlockingConnection(pika.ConnectionParameters("rabbitmq")) File "/usr/local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 360, in __init__ self._impl = self._create_connection(parameters, _impl_class) File "/usr/local/lib/python3.10/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection raise self._reap_last_connection_workflow_error(error) pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)

Status: machine: minikube pid: 391461 route: 10.96.0.0/12 -> 192.168.49.2 minikube: Running services: [] errors: minikube: no errors router: no errors loadbalancer emulator: no errors
This is what logs continuously in my stdout. Someone please help!