adjunct Manager - Spark Developer:Job description:
Job Description »AM, Spark Developer » Should have practice working on Spark and SQL modules of Spark broadly » practice in designing and developing applications in Spark using Scala to compare the performance of Spark with Hive and SQL/Oracle. » Should have practice in analyzing Hive SQL scripts and crafted a solution to implement using Scala » Should have practice in crafting and developing applications in Spark using Scala to compare the performance of Spark with Hive and SQL/Oracle. » Develop Spark scripts by using Scala shell commands as per the requirement » Develop Scala scripts, UDFs using both Data frames/SQL/Data sets and RDD/MapReduce in Spark for Data Aggregation, queries and writing data back into OLTP system through Sqoop » Optimizing of existing algorithms in Hadoop using Spark Context, Spark-SQL, Data Frames and Pair RDD's. » practiced in performance tuning of Spark Applications for setting right Batch Interval time, accurate level of Parallelism and memory tuning » skills with the tools in Hadoop Ecosystem including Hive, HDFS, MapReduce, Sqoop, Spark, Kafka, Yarn » Excellent knowledge on Hadoop ecosystems such as HDFS, Job Tracker, Task Tracker, Name Node, Data Node » Very good knowledge of Partitions, Bucketing concepts in Hive and crafted both Managed and External tables in Hive to optimize performance. » practiced in handling large datasets using Partitions, Spark in Memory capabilities, Broadcasts in Spark, Effective & efficient Joins, Transformations and other during ingestion process itself » Develop Hive queries to process the data and generate the data cubes for visualizing » Implemented Partitioning, Dynamic Partitions, Buckets in HIVE. » Involved in converting Hive/SQL queries into Spark transformations using Spark RDDs, and Scala. » practice in manipulating/analyzing large datasets and finding patterns and insights within structured and unstructured data » Should be involved in building Hive tables, and loading and analyzing data using hive queries » Proven practice with SQL queries and database tuning » Strong knowledge of database design and development with previous practice in developing ETL processes, and multifaceted data models » Respond to & solving support enquiries from users across various groups including Finance, Digital and Operations Qualifications we seek in youMinimum Qualifications » Programming Required languages Java, C, C++, Scala, Python » Scripting: Shell » Operating systems: Linux, Unix, Windows » RDBMS/No SQL: SQL Server, MySQL, Oracle 11g, Azure, HBase » Hadoop Ecosystem: Spark Scala, Spark Python, Map Reduce, Hive, HBase, HDFS, Sqoop, Pig, , Zookeeper, Kafka, Spark streaming, Oozie Preferred qualifications » Analytical thinking and problem solving abilities. » Good Presentation Skills, Other details » Department: General / Other Software General / Other Software » Business Sector: BPO / Call Center BPO / Call Center » Special Skills: sales, sales, mis, mis, accounts, accounts, tat, tat, banking, banking, sql server, sql server, oracle 11g, oracle 11g, data cubes, data cubes, sql queries, sql queries, customer focus, customer focus, problem solving, problem solving, database design, database design, data aggregation, data aggregation, performance tuning, performance tuning, c, c, go, go, c c Recruiter details »Employer's name Genpact »Company Description: GENPACT India Private Limited View similar jobs All Jobs by this recruiter
Skills:
|
||||||||||||||||||||||||||||
|