Turning Sensor Chaos into Data Clarity
By Ciaran Kirk, Operations Director IMGS & DataBuilders
Introduction:
Integrating sensors into facility management is transforming how buildings are managed. Sensors play a crucial role in Facilities Management (FM) by making buildings smarter, safer, more efficient, and cost-effective.
Sensors and IOT devices provide real-time data that helps managers make better decisions instead of relying on manual checks or guesswork. They can be used for everything from space utilisation to environmental monitoring.
The challenge for organisations is that sensors generate a lot of data which can provide great insights but it is very easy for this big data to create a data swamp and provide less clarity!
In this blog we will outline how our DataBuilders solutions can deliver an end-to-end sensor to insight data pipeline.
Sensor Integration with Safe Software FME
FME can transform data from sensors and IoT devices into information you can act on. FME provides a number of methods to connect to Sensor or IOT devices via a range of protocols:
IoT Gateways
IoT Gateways are intelligent central hubs for IoT devices. Gateways connect IoT devices that have limited compute and storage to the cloud.
FME provides a number of connectors to IOT gateways including AWS, Azure, Google and IBM.
Message Broker
FME supports message brokers typically used to connect to IoT data sources, such as Kafka, MQTT, and RabbitMQ.
For example MQTT is a standards-based messaging protocol, or set of rules, used for machine-to-machine communication. IoT devices typically have to transmit and receive data over a resource-constrained network with limited bandwidth.
In these cases MQTT is used for data transmission, as it is easy to implement and can communicate data efficiently. MQTT supports messaging between devices to the cloud and the cloud to the device.
FME provides the MQTTConnector for devices using this protocol.
Rest API
Real-time data can also be delivered using REST APIs as the interface protocol. The rest API will enable you to stream the data continuously to your destination.
To help process the data it may make sense to break data into time based groups otherwise know as windowing. This can be done using the TimeWindower transformer.
Windowing means taking the data stream and, using time, breaking the stream up into groups. Once in groups, the data is ready for analysis.
Once FME can connect to the sensor via the preferred protocol the data can be validated in FME before being wrote to database for storage. FME can also restructure and enhance the data to prepare the data for analytics.
Data Storage
With FME supporting so many database writers we can choose from a wide variety of data warehouses to store the sensor data. But we have found Amazon Relational Database Service (Amazon RDS) is a very scalable, secure data warehouse.
Amazon RDS is an easy-to-manage relational database service optimised for total cost of ownership. It is simple to set up, operate, and scale with demand so the perfect solution for analytical warehouses. The PostgreSQL version of RDS provides a very cost-effective option with a free tier available for development purposes.
There are lots of other alternatives to PostGreSQL that we can choose from. For example Snowflake now also provides the ability to run remote engines in Snowflake. FME Remote Engines run directly within Snowflake using Snowpark Container Services, allowing data to be processed in place without movement for greater security and efficiency.
Data Preparation
With FME we stream the data updates from the Sensors to the data warehouse as they happen but FME can also aggregate the data to optimise data storage.
For complex analytics a combination of the database design and the data load process can be used to prepare the data at the model level.
On projects we have worked on, one of the key tasks is to pivot the sensor data from column to rows. This make its easier to run analytics e.g. for total hours in a room is in use because you don’t have analyse each column (for example if a sensor is pinging every 10 minutes that would require over 140 columns).
As part of the data transformation we can also enhance the data and provide meaningful values e.g. if the sensor is monitoring room occupancy instead of a 1 or 0 for the room being in use we could have an actual value i.e. free or in use:
This will make the dashboard designers job easier when it comes to creating dashboards which will then naturally decrease time to insight.
Embedded Insights
Once we have our data warehouse populated with the Sensor data and the data structured properly in the warehouse, it is very easy using Sisense to create the dashboards to provide the insights.
The benefit of using Sisense to deliver the insights is that the dashboards can be easily white labelled to the facility management companies branding.
In a lot of cases facility management companies provide a software app to their customers. With Sisense it is very easy to embed the dashboards into these applications without occurring large development costs by developing analytics from scratch.
Summary
Using analytics in facility management helps turn building data into smart decisions that save money, improve comfort, and increase efficiency.
With the DataBuilders platform facility managers can see a true return on their investment in Sensor and IOT devices.
