🎧 Industrial Data Quality Podcast E1: Introduction
Welcome to my podcast! In this very first episode I introduce the topics of this podcast and explain my background in data. Follow the show About Denis Gontcharov
🎥 Testing Data Quality with Soda Core in Databricks
In this video I demonstrate how to perform data quality checks on a Delta table in Databricks using Soda Core. Soda Core is the open-source Python package developed by Soda. It can be compared to Great Expectations, but is much simpler in my opinion. I enjoy using Soda in my professional projects and will continue exploring this framework.
Hosting Great Expectations Data Docs on Azure Blob Storage
Resources Check out the complete code on GitHub. Browse the GX Data Doc on Azure Blob Storage. Use Case Last week I explored Soda as a data quality testing framework for my large enterprise client. This week I’m exploring a more mature alternative called Great Expectations or GX in short. GX generates neat HTML reports called Data Docs that give an overview of your data quality test results. The client wants to share these reports with the team - but not with the world!...
Exploring Soda Data Quality Testing on Databricks
Use Case For my current engagement I’m tasked with developing an automated data quality framework for a large industrial enterprise in the renewable energy sector. The client has over a hundred independent SCADA systems from various vendors gathering energy production data. All this data has to flow in one central repository to be analyzed with Databricks. The client is obligated to ensure high data quality for contractual reporting to external parties....
Developing a Qtile VPN Widget
I often work from WeWork, which is a public co-working space with a shared WIFI-network. Securing my connection For security and privacy reasons, I therefore work behind a VPN. I purchased an AirPVN subscription and configured it as an OpenVPN connection in NetworkManager1: NetworkManager’s terminal user interface nmtui This way, even if I’m working from Prague, my location will always be shown as Belgium2. This was important because my current enterprise client’s IT-security team kept receiving alarms that my location was changing all over Europe....
🎧 Smart Metals Podcast E13: From the Shop Floor to the Cloud: AI in Metal Manufacturing
In this episode of the Smart Metals Podcast Denis and I, dive into the topic of transferring shop floor data to the cloud and leveraging AI for predictive maintenance and other use-cases in the metals industry. We discuss essential components like the Unified Namespace (UNS) and Data-Centric AI, highlighting why smaller manufacturers shouldn’t shy away from cloud technology. We also break down common misconceptions about AI, particularly for SMBs, and explore the benefits of cloud services....
🎧 Smart Metals Podcast E12: Navigating the Future of Factory Connectivity with Russ Waddell
This week at Smart Metals Podcast, we, the hosts Denis Gontcharov and Luke van Enkhuizen, had a fascinating conversation about the future of factory connectivity with Russ Waddell, an expert in industrial connectivity. We explored the challenges of integrating manufacturing systems, the importance of cultural and technological shifts, and how to leverage AI and data science to unlock the full potential of manufacturing processes. We delved deeper into topics like the evolution of industrial connectivity, unified namespaces, and strategies for starting your digital transformation journey....
🎧 Smart Metals Podcast E11: Unifying Manufacturing with Unified Namespace: A Conversation with Brian Pribe
This week at Smart Metals Podcast, we, the hosts Denis Gontcharov and Luke van Enkhuizen, had a conversation about implementing Unified Namespace (UNS) in manufacturing environments with Brian Pribe, founder and president of MACH Controls LLC. We explored how to make advanced manufacturing technology accessible to businesses of all sizes. We delved deeper into practical implementation of UNS, the combination of MQTT and Kafka, and making these solutions affordable for smaller manufacturers....
🎧 Smart Metals Podcast E10: Unifying Manufacturing with Unified Namespace: A Conversation with Brian Pribe
This week at Smart Metals Podcast, we, the hosts Denis Gontcharov and Luke van Enkhuizen, had a conversation about implementing Unified Namespace (UNS) in manufacturing environments with Brian Pribe, founder and president of MACH Controls LLC. We explored how to make advanced manufacturing technology accessible to businesses of all sizes. We delved deeper into practical implementation of UNS, the combination of MQTT and Kafka, and making these solutions affordable for smaller manufacturers....
🎧 Smart Metals Podcast E9: A Unified Approach for Industry 4.0
In this episode of the Smart Metals Podcast, hosts Denis Gontcharov and Luke van Enkhuizen welcome Todd Abraham from InnoVaas. They dive into digital transformation within the metals industry, focusing on the Unified Namespace’s growing influence in Asia and Europe. The discussion covers enabling technologies like Apache Kafka and MQTT, cultural differences in technology adoption, and how agile approaches help smaller companies innovate. The episode concludes with advice on navigating digital transformation and the importance of structured data for machine learning....