ETL is used by the Data Engineers, ETL or ELT Developers and involves the learning process for various tools and the architecture’s used. With the emergence of the big data and different formats , the ETL process used for data migration becomes sometimes complex. The purpose of this web site is to discuss various scenarios… Read more

Lambda Architecture : Presents holistic view of an organization’s data , from historic and real-time view. For workload or data load processing and presentation of the big data Nathan Marz used the term Lambda Architecture (LA). Linear Scalable : The Architecture should scale out and should take care of different types of use cases. Fault… Read more

Move the files in the SFTP location to another folder after they are copied , with a Timestamp for the Date the File is copied. Problem : Load the data from csv files in a SFTP location into the azure data warehouse. The source Folder is in a SFTP location (AWS Cloud VM) , where… Read more