Spark ETL process

Hi, I am cloudera certified developer and worked on each component which you described. Please let’s connect and discuss more on your requirements.

Hi, I have 8 years of experience and working on hadoop, spark, nosql, java, BI tools(tableau, powerbi), cloud(Amazon, Google, Microsoft Azure)... Done end to end data warehouse management projects on aws cloud with hadoop, hive, spark and presodb. Worked on multiple etl project like springboot, angular, node, PHP, Kafka, nifi, flume, mapreduce, spark with XML/JSON., Cassandra, mongodb, hbase, redis, oracle, sap hana, ASE.... Many more. Let's discuss the required things in detail. I am committed to work done and strong in issue resolving as well. Thanks

Hi i am a Big data spark developer. I can help you on the project, please discuss about the project, for better understanding.

I am working as a data Engineer in mnc. I have great knowledge of big data,spark, pyspark and python. let me know if we can connect.

Hi, I have an experience of 5 years and worked on several different projects which includes HDFS, HIVE, SPARK and SCALA, KAFKA My bid includes development + testing ( Along with the unit test case document) Let me know the requirement and we can proceed a head with this project.

I have 7+ years experience in Big Data involving the Hadoop stack. ETL pipelines in Cloud or on premise using the Big Data Platform. Spark streaming and Batch is both comfortable for me. Experienced in designing big data architecture and implementing the same. Apache Spark implemented in Scala programming. IDE used is IntelliJ but also comfortable in Eclipse.

My proposal is to help you make the data ingestion pipe line from kafka to apache hadoop and then process them with apache spark, using the pyspark and spark sql library

Hi I am an AWS Certified and Big Data Developer and has worked on all the skills which are mentioned in project details and has good command in programming languages along with ETL projects. Lets connect and discuss more about project, would love to work with you. Thanks

4 years of industry big data experience.

• Utilized Apache Spark with Python to develop and execute Big Data Analytics. • Strong experience in the Analysis, design, development, testing, and implementation of Business Intelligence solutions using Data Warehouse/Data Mart Design, ETL, OLAP, BI. • Strong understanding of Data Modeling (Relational, dimensional, Star and Snowflake Schema), Data analysis. • Implemented the complete Big data pipeline with real-time processing. • Worked with Sqoop to ingest & retrieve data from various RDBMS like Oracle DB & MySQL. • Worked rigorously with Impala 2.8.x for executing ad-hoc queries • Written SparkSQL queries to transform data for further downstream processing • Preparing HLD and LLD (Lower Level Documents) like Source to target data loading process. • Analyzing existing customer satisfaction functionality requests for providing business-value based recommendations to Product Owners • Facilitating requirement and root cause analysis and other requirements workshops • Writing complex queries to fetch data from different data sources in an optimized manner.

I have 4.5 years of relevant hands on experience on big data using spark and scala and most of the projects I have worked were on ETL

Hey I have 10+ year of exp in etl and big data(spark, python, hive etc). I work in USA and currently in India and can manage this in duration. I am looking for min 2500 per 8 hrs for quality work.

Hi, I read your requirements and I firmly believe that I can be of help here. Let's discuss this further on chat, to explore how may I help you in this. Looking forward to work with you. Regards, Fahad

• Data Scientist executing data-driven solutions to increase efficiency, accuracy, and utility of internal data processing. • Experienced at creating data regression models, using predictive data modeling, and analyzing data mining algorithms to deliver insights and implement action-oriented solutions to complex business problems. • Able to assess business rules, collaborate with stakeholders and perform source-to-target data mapping, design, and review. • Solutions-oriented Systems Analyst possessing extensive experience and a unique combination of skills, including Project Analysis, System Analysis, and Software Quality Assurance. Hadoop Ecosystem Framework - Hadoop HDFS, Hadoop MapReduce, Yarn Query/Data Flow - Hive, Cassandra Data Access - HBase, MongoDB Co-ordination - Oozie, Zookeeper Statistical Tool - R, Python, Scala Data Ingestion - Flume, Sqoop Data Transformation and Analysis - Scala, Pig, Hive NoSQL Database Tools - HBase, MongoDB Machine Languages -R, Python Spark Components Spark SQL, Spark Streaming, MLib, GraphX

Hi, I am working in big Data technologies like Hadoop,Spark and Kafka since 6 years. I m well versed with all the technologies that mentioned. We can talk if you are interested. Thanks.

I have 2.5 year out of 10 year of IT experience in spark with python. I can create ETL process in spark with python. I am also certified developer of spark and hadoop. Thanks, Ankur

Hi I am big data engineer and have very good experience in big data ecosystem, give spark Streaming with kafka in Java/Scala languages. if interested will talk on further details

As i mentioned in my introduction, i have good experience in Spark and spark SQL by building applications for ML models while working full time as data engineer for 1 year and have good hands on of the language. Know the intricacies of optimizing performance and administrating the ambari to improve resource usage.

• A result-oriented professional with nearly 7 years of experience in Telecom and Oil Production Sectors • Capabilities in using analytical processes and techniques to improve and optimize information and decision systems • Good understanding/knowledge of Hadoop Architecture and its components such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node, Secondary Name node, and MapReduce concepts. • Developed Pig Latin scripts for extraction of data from server and loading into HDFS for data cleansing and analysis. • Experience in performing analytics on structured and unstructured data using Hive queries. • Experience in Big Data ecosystem and its various components such as MapReduce, HDFS, HIVE, HBase, PIG, Sqoop, Zookeeper, Oozie and Flume • Hands on experience in writing Spark programs using features of RDD, spark SQL, datasets and dataframes.

Designated as a Big Data Engineer with 5+ demonstrated years of experience in the Hadoop world. Delivered multiple Proof Of Concepts on for integrating different tools in Hadoop Stack for transforming a huge volume of Data. Real-time exposure in handling multiple Datalake applications in production. Strong knowledge in Batch and Real-time data Data Integration. Involved in high-level discussions with Data Architects on arriving the most efficient and performant solutions. I have hands-on experience in integrations on creating ETL workload with different source system like RDBMS (Especially Oracle), NoSQL Data stores and also worked on integrating Kafka with Spark Structured Streaming (Both on Producer and Consumer processes).