In this video I explain how to read data from an Airtable into a PySpark dataframe on Databricks. Airtable is a popular spreadsheet tool used at many enterprises. It offers additional features compared to other tools such as Microsoft Excel and Google Sheets
In this video I demonstrate how to perform data quality checks on a Delta table in Databricks using Soda Core. Soda Core is the open-source Python package developed by Soda. It can be compared to Great Expectations, but is much simpler in my opinion. I enjoy using Soda in my professional projects and will continue exploring this framework.
This video was recorded by me for the United Manufacturing Hub In this video, we’ll walk through connecting HighByte, a data processing engine, with the United Manufacturing Hub (UMH). HighByte is an OT-friendly alternative to Node-RED to handle operational technology data, making it a useful tool alongside the UMH. We will start by demonstrating how to download and set up HighByte within Kubernetes. Next, we’ll show you how to configure it to use data from the UMH, model this data, and send it back to UMH....
This video was recorded by me for the United Manufacturing Hub In this first part of the UNS Basics mini series, we introduce the Automation Pyramid as a system to control the plant. We highlight the challenges related to data analytics and present the Unified Namespace as the solution. We also show how the Automation Pyramid connects to the Unified Namespace, according to the United Manufacturing Hub’s philosophy. Finally, we briefly discuss where historical data from the Unified Namespace is saved....
This video was recorded by me for the United Manufacturing Hub In this tutorial, we guide you through the process of integrating Ignition by Inductive Automation with the United Manufacturing Hub (UMH). The objective is to build an operational dashboard in Ignition that displays real-time MQTT data coming from the UMH, using UMH as a Unified Namespace (UNS). This tutorial requires a running UMH instance, and (at least) the demo version of Ignition by Inductive Automation....
This video was recorded by me for the United Manufacturing Hub In this tutorial, we guide you through the process of building a Grafana dashboard based on the PackML model. We begin with a look at the PackML MQTT simulator, which simulates a real production line using the PackML standard and integrates with MQTT for messaging. Following this, we build a dashboard that can display various machine states (such as Starting, Running, Stopped, etc....