André Ichiro

Hi there! I am a data engineer and continuous learner, driven by a passion for leveraging data to drive meaningful insights and solutions. My focus is on designing and implementing ETL and ELT data pipelines and managing the end-to-end data lifecycle.

Transformations and Analysis:
Skilled in Python, Pandas, Spark, Pyspark, dbt, and Seaborn for data transformations and insightful analysis.

Cloud:
Utilized various AWS services such as Glue, S3, Lambda, Kinesis, documentDB, ECS, Fairgate and API Gateway, alongside with Databricks and Snowflake for analytics and data processing.

Data Storage:
Working in SQL, Postgres and MongoDB databases to implement data storage solutions.

DevOps:
Familiarity with CI/CD practices, containerization using Kubernetes and Docker, version control with Git & GitHub, and infrastructure automation with Terraform.

Messaging Systems:
While my experience with Kafka is limited, I have some exposure to it for real-time data streaming and event-driven architectures.

Machine Learning:
I have a foundational understanding of PyTorch, TensorFlow, exploratory data analysis and basic supervised, semi and unsupervised learning techniques, as well as recommender systems.

Back-end & Web:
I have gained experience in developing back-end systems using Python, Django, Django REST API, and following software engineering principles.

Architecture and Best Practices:
Adhere to industry best practices such as Pep 8, SOLID, and Domain Driven Design principles.

Background:
With a background in computer science, law (+10 years of experience in international private law and BA's degree with honors) and entrepreneurship (product, marketing, sales), I bring a unique perspective to problem-solving with the ability to collaborate effectively in interdisciplinary environments

In my most recent personal project, I developed an automated ELT data pipeline using a variety of tools and technologies including multiple AWS Services, Spark, Python, Pandas, Terraform, Docker, Ansible, Airflow, Databricks, GitHub Actions and dbt cloud. This project provided insightful video analysis and trained a Longformer Language Model (LLM) using YouTube and Whisper transcriptions.

Competências: Terraform · Databricks · Infrastructure as code (IaC) · Data Lake · AWS Glue · AWS Athena · AWS Lambda · Amazon Kinesis · dbt · AWS API Gateway · Data Warehousing · Data Engineering · Data Modeling · Docker · Pandas (Software) · Apache Spark · Ansible · GitHub · PySpark · Data Pipelines · Apache Airflow · SQL · Extract, Transform, Load (ETL) · Python · Design Patterns · DevOps

Universidade Presbiteriana MackenzieUniversidade Presbiteriana Mackenzie

Bachelor's degree, Computer Science