Streaming large files directly from SFTP to S3 without any intermediate disk storage, using Python's asyncio
- Published On
Stay Updated with Latest Posts
Join the newsletter and never miss insights on data engineering, machine learning, and analytics.
We respect your privacy. Unsubscribe at any time.
Shaurya's Blog
Data, ML and Software Engineering
- Enriching a stream using an external database with Apache Flink
- Published On
- Consuming CDC logs from a backend database to an Iceberg table using Flink
- Published On
- In this post, I'll cover the basics of networking in AWS, including VPCs, subnets, security groups and route tables that every data engineer should be familiar with.
- Published On
- This is a series of blog posts about distributed data-intensive systems.
- Published On
