Using GoLang in Data ExtractionIn the fast-paced world of data engineering, where large volumes of data need to be ingested from many different sources, performance and…Dec 27, 2024Dec 27, 2024
GitHub Action — Build, Push Docker Image to Artifact RegistryThis blogpost is going to introduce a public GitHub action I wrote that builds and pushes a Docker Image to Google Cloud’s Artifact…Nov 9, 20241Nov 9, 20241
Example of Kestra GCP Batch Task RunnerKestra GCP Batch Task Runners is a powerful Kestra feature for running batch jobs on Google Cloud Platform (GCP). It provides a scalable…Sep 9, 2024Sep 9, 2024
Why We Switched Integrations from Python to GoIn the world of growing data, you always need to have top-of-the-line and efficient fetching integrations. At Foundation Direct, we need…Aug 14, 2024Aug 14, 2024
How to Extract a Dataset from BQ to GCS using KestraEfficient/easy-to-build automations are a necessity. In this post, we will build a BQ-to-GCS automation with Kestra!Jul 22, 2024Jul 22, 2024
Kestra.io — Powerful Declarative WorkflowsThe future of orchestrating workflows… the declarative way!May 17, 2024May 17, 2024
A Beginner’s Guide to Getting Started with Mocking in PytestAs an Engineer you will be expected to know how to write and develop independent Unit Tests. You are able to do this with Pytest-Mock.Apr 10, 20241Apr 10, 20241
Improve Code Quality with Flake8Writing clean, readable, and PEP 8-compliant code is what Python programmers strive for. It’s not only “following the best practices” but…Dec 28, 2023Dec 28, 2023
An Overview of Cloud Run Jobs and PrefectA brief overview of Cloud Run Jobs and Prefect.Dec 20, 2023Dec 20, 2023
