rafaeltedesco / azure-storage-file-system-full-stack Goto Github PK

I'm Rafael Tedesco, a seasoned Software and Data Engineer with over a decade of experience specializing in planning, building, and maintaining resilient and scalable web applications, and automating processes for data extraction, transformation, and loading (ETL) in Cloud environments. I support business specialists in their decision-making by providing robust data solutions.

What I Bring to the Table:

🚀 Extensive Experience in Data Engineering: Proven track record in developing ETL/ELT automation for data engineering, web applications, and analytics.

🔧 Proficiency in Diverse Technologies: Skilled in Python, SQL, NoSQL, and various cloud services including AWS and Azure.

📊 Building and Optimizing ETL Pipelines: Expertise in creating efficient ETL pipelines for data extraction, transformation, and loading.

🌐 Cloud Expertise: Extensive experience with AWS (Lambda, Glue, Athena, S3, Rekognition, SageMaker, and more) and Azure (Databricks, Data Lake Storage).

📚 Continuous Learning: Always exploring new technologies, development patterns, and best practices to stay ahead in the field.

Key Skills and Technologies:

• Programming Languages: Python
• Types of Database: SQL, NoSQL
• Frameworks and Libraries: Pandas, PySpark, Pydantic, SQLAlchemy, NLTK, Flask, FastAPI, Django, Django Rest Framework, Streamlit, Jinja2, SQLModel, Beautiful Soup, Pillow, Selenium, PyTesseract (OCR Tool), PyPDF2, PyMuPDF, Python-Docx, Boto3, langChain
• Cloud Services: AWS (Lambda, Glue, Athena, S3, Rekognition, SageMaker, etc.), Azure (Databricks, Data Lake Storage Gen2)
• Architecture and Design Patterns: Medallion Architecture, Clean Architecture, Hexagonal Architecture, DDD, TDD, SOLID, OOP Design Patterns, Microservices Design Patterns
• Big Data and Data Streaming: AWS Firehose, RabbitMQ, Kafka, SQS/SNS, RabbitMQ
• Containers: Docker
• BI Tools: Power BI (DAX, Power Query, Data Modeling), Excel
• Machine Learning and AI: Hugging Face, OpenAI, LLM, RAG, scikit-learn
• Other Technologies: Redis, Excel, VBA, GraphQL, JWT, Oauth2, Vector Databases (ChromaDB), SAP Netweaver