Simple Example Funnels

Let's imagine we have a simple analytic - we'd like to record a differential temperature (the difference between two temperatures) from ARDI to a MySQL database.

In this case, we can identify the source (ARDI) and the sink (MySQL).

From our filters, we need to find a suitable filter to calculate the difference. It just so happens that we have a filter named Difference that does the job for us. It takes all of the input values and returns the difference between the maximum and the minimum value of that row.

So we create the following funnel using XML…

<root>	
	<funnel name="DiffTemperature">
		<source key="ardi"></source>		
		
		<channels>			
			<query>('Machine 1','Machine 2') ASSET 'Temperature' PROPERTY VALUES</query>
		</channels>
		
		<filter type="Difference">
                      <column>MachineTempDiff</column>
                </filter>	
		
		<filter type="Change">
		</filter>
			
		<filter type="SQL">
			<table>recordings</table>
			<columns>name,stamp,value</columns>
			<prefix>'difftemp',NOW()</prefix>
		</filter>			
		
		<sink key="mysql"></sink>
	</funnel>
</root>

Channels defines the measurements we are interested in loading from the source.

The Difference filter returns the difference between the minimum and maximum value of each row.

The Change filter stops the funnel from completing unless there is a change to the output data. This prevents the analytic system from storing/sending redundant duplicate data.

The SQL filter creates a SQL insert string based on the incoming data frame that can be sent to SQL-compliant sinks.

Now we'd like to calculate a rolling average of our temperature from ARDI. In this case, we'd like to send the information to Microsoft SQL Server and REDIS.

In this case, we can identify the source (ARDI) and the sinks (MSSQL and Redis).

In this case, Mean is appropriate. It takes all of the input values and returns the mean of each column.

But we have an issue - how do we make this the mean of 30 seconds of data?

This is where buffering comes in.

We can request that Analytics keeps a certain amount of data in a buffer to be processed. In this situation, we can specify a fixed value of 30s.

So we create the following funnel using XML…

<root>	
	<funnel name="RollingPressureAverage">
		<source key="ardi"></source>		
		
		<channels>			
			<query>('Machine 1') ASSET 'Pressure' PROPERTY VALUES</query>
		</channels>

                <buffer type="fixed" dedup="false" seconds="30"/>
		
		<filter type="Mean">                      
                </filter>	
		
		<filter type="Throttle">
                      <duration>30</duration>
		</filter>						
		
		<sink key="redis"></sink>
	</funnel>
</root>

Buffer sets the funnel buffer to a fixed 30s of data, with no de-duplication.

The Mean filter calculates the average of all of the rows, resulting in a single row containing the average of every column.

The Throttle filter ensures that the funnel only executes a maximum of once every 30 seconds, reducing the amount of redundant data storage.

A common question is “how long was the machine running?”, or “how long was it in this state?”

Using analytics, you can convert your discrete values to time frames, which changes this…

To this…

<root>	
	<funnel name="MotorRunning">
		<source key="ardi"></source>		
		
		<channels>			
			<query>('Machine 1') ASSET 'Running' PROPERTY VALUES</query>
		</channels>

                <buffer type="dynamic" dedup="false"/>
		
		<filter type="DiscreteFrame">                      
                </filter>	
				
		<filter type="SQL">
			<table>frames</table>
			<columns>name,value,fromtime,totime</columns>
			<prefix>'motorrun'</prefix>
		</filter>			
		
		<sink key="mysql"></sink>
	</funnel>
</root>

Buffer sets the funnel buffer to be dynamic (infinite length & purged on output), with deduplication.

The DiscreteFrame filter stops the funnel from outputting until the value changes, and radically alters the content of the data frame to include 'start' and 'end' times.

As the DiscreteFrame filter only produces output when the state of the input changes, there's no need to put any filters in to control output rate (such as Throttle or Change_.