Senior Manager - Data Science & Engineering @ Niyo | ex HSBC, StanC | MDI Gurgaon, ESCP Business School
My blog post on "Building a Data Science Platform with Kubernetes" was published in Towards Data Science, explaining how to set JupyterHub, MLflow, and SeldonCore in Kubernetes https://lnkd.in/gtaf6HQY #TowardsDataScience #DataScience #Kubernetes #JupyterHub #SeldonCore #MLflow
successfully participated in the master class of data science at the Scaler platform. it is the best platform which provides the better roadmaps. I got best knowledge from here. #scaler #datascience
🚀 Delighted to announce my completion of the 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗲𝗲𝗿𝗶𝗻𝗴 𝗭𝗼𝗼𝗺𝗰𝗮𝗺𝗽 by DataTalksClub! Thanks Alexey Grigorev. 🛠️ Covered: #Docker: Containerization #Terraform: Infrastructure #Mage: Workflow Orchestration #BigQuery: Data Warehousing #DBT: Data Transformation #Spark & #Kafka: Batch and Stream Processing 🏆 Successfully built an end-to-end data pipeline. Excited for the next chapter in my data journey! 📊💡 #DataEngineering #DataTalksClub #ContinuousLearning
Introduction to Data Structure and Algorithms masterclass from Geekster #geekster #masterclass #dataStructure #algorithms
Hello Linkedin Community! 👋 I am super excited to share that I have completed the Data Science Bootcamp with HyperionDev! Thanks to all the lecturers and mentors at HyperionDev for being supportive through this incredible journey. #datascience #hyperiondev #dataanalytics #upskilling
Application Consultant at ACS Infotech Pvt. Ltd. | Big Data Enthusiast | SQL | Python | Hadoop | PySpark | Data Engineering & Analytics
🚀 Let's dive into some crucial concepts from Week 6 Spark Dataframe Transformations from Sumit Mittal sir’s ULTIMATE BIG DATA MASTER PROGRAM! 🚀 🔹 Schema Enforcement: Ensuring data consistency and quality by enforcing predefined schemas (Schema ddl & StructType). 🔹 Different Types of Read Modes in DataFrame: Managing how Spark handles corrupt or malformed records with PERMISSIVE, DROPMALFORMED, and FAILFAST modes. 🔹 DataFrame Creation: Techniques for creating and manipulating DataFrames efficiently. 🔹 Concept of Nested Schema: Handling complex data structures within DataFrames to maintain data integrity using Schema Enforcement. 🔹 Select vs SelectExpr: Exploring the differences and use cases for select and selectExpr for data transformations. Thank you to my mentor, Sumit Mittal, and the amazing TrendyTech team for their unwavering support and guidance. Here’s to more learning and growth in the world of big data! 🌟 #DataEngineering #BigData #ApacheSpark #DataFrames #DataQuality #SchemaEnforcement #ETL #DataScience #LearningJourney #TrendyTech
From Non-Tech to Data Science Enthusiast: 🌟 Excited to have participated in the masterclass on 'Real-world problem-solving using data science techniques' as part of my journey with Scaler DSML course. 💡 Every step forward is a leap towards unlocking new opportunities in the dynamic world of data science! 🚀 #datasciencejourney #problemsolvingskills #neverstoplearning #scaler #dataanalytics #datascience #businessintelligence
Data Engineer at Tata Consultancy Services || ETL | SQL | PySpark | Python | Hadoop | HDFS | Hive | Sqoop | Data Modelling | Git and BitBicket
Exploring the Power of PySpark Features 🚀 From resilient distributed datasets (RDDs) to DataFrame transformations and MLlib, PySpark offers a robust toolkit for big data processing. #PySpark #BigData #DataEngineer #tataconsultancyservices
#LoomeConnects to #ApacheSpark Extracting, Loading and Monitoring Data in #SparkSQL has never been easier. Discover how Loome allows orchestration of data pipelines across data engineering, data science and high performance computing workloads with native integration of Spark SQL #DataPipeline tasks. Loome allows you to easily monitor Data Quality exceptions and keeps an audit trail of resolved issues, proactively managing data quality with a fully automated data quality engine generating audience targeted alerts in real-time. #DataWarehouse #DataQuality #SQL #DataLake #DataScience #HPC 👉 https://bit.ly/3IksWM9
#LoomeConnects to #ApacheSpark Extracting, Loading and Monitoring Data in #SparkSQL has never been easier. Discover how Loome allows orchestration of data pipelines across data engineering, data science and high performance computing workloads with native integration of Spark SQL #DataPipeline tasks. Loome allows you to easily monitor Data Quality exceptions and keeps an audit trail of resolved issues, proactively managing data quality with a fully automated data quality engine generating audience targeted alerts in real-time. #DataWarehouse #DataQuality #SQL #DataLake #DataScience #HPC 👉 https://bit.ly/3IksWM9
Just finished the course ‘Hands-On Introduction: Data Engineering,’ instructed by Vinoo Ganesh, CEO and founder of Stealth Startup. This course covers Data Pipeline Role and Responsibilities, Orchestration and ETL Patterns, and Airflow Concepts. #tataelxsi #dataengineering #training #linkedinlearning
