Writing a Report in Python and MatPlotLib

This article is not a complete tutorial on using Pandas and MatPlotLib. These two libraries are extremely useful for creating reports, but they are also rather large and complex topics to cover in a small space.

NOTE: The code below will still work, but you might find it even easier to use our new MPL Reporting Library, which has a number of time-saving features.

We are just going to illustrate a basic example of producing a simple line graph that can be emailed out.

We usually add the following line at the start of each script…

import os
import sys

sys.path.insert(0,os.path.dirname(os.path.dirname(__file__)))

import matplotlib
import pandas as pd
import aql
import templateblock

These lines add the base reports directory as a source of module files, so that you can create shared code between each of your reports.

MatPlotLib is the charting library we will be using, and Pandas is an excellent system for working with large quantities of tabular data.

The AQL and TemplateBlock modules are provided by Optrix, and allow you to quicky get reports up-and-running.

The addon calls your report with a set of arguments. You can parse and handle all of thse using the ReportArgs function in the aql module.

args = aql.ReportArgs("My Report Name")

The args variable contains quite a bit of information about the requested report - particularly the starting and ending times as UTC strings.

ststr = args.start
etstr = args.end

fig, ax = plt.subplots(1)
fig.set_size_inches(11.69,8.27)

The first line creates our plot axes - in this case, our plot only has one set of axes, containing our line chart.

We then set our report to be an A4 landscape paper size.

In this case, we are going to be producing a simple line graph of the Temperature of asset TE104 during the day.

We need to grab the data.

   results = aql.query(server,"('TE104') ASSET ('Temperature') PROPERTY VALUES { \"grain\": -512, \"start\": \"" + ststr + "\", \"end\": \"" + etstr + "\" } GETHISTORY")

   df = aql.historyToDataFrame(results,report=args)

The first line requests the temperature history (across 512 points) from ARDI and returns it as a JSON collection.

The second line then converts that JSON collection into a Pandas dataframe.

The final format of the dataframe is below…

The dataframe will have a column for each valid property in the AQL result. The column name will be the asset name followed by a space, followed by the Property Name

Your data is likely to have holes, caused by different sample rates between different channels and by sensors occasionally being bad/offline.

You should take the opportunity to clean your data here.

    df.fillna(method='backfill',inplace=True)
    df.fillna(method='ffill',inplace=True)

The code above does a backfill and forward fill on your data, making sure you don't have any stray undefined data in your data frames.

You might also want to consider using df.interp on your analogue values if preferred.

    ax.plot(df['TE104 Temperature'], linewidth=1, linestyle='-.', color='b',label='Temperature')
    ax.margins(x=0,y=0)
    ax.grid()
    ax.legend()

The first line draws the actual plot, as a dashed line in blue.

The second line removes any padding that the chart may have defaulted to.

The third line adds a simple grid to the background of the report.

The fourth line adds a dynamically-generated legend to the report. Fairly useless for a single-channel report like this, but it will come in handy if you add new measurement channels.

    fig.tight_layout(rect=[0, 0, 0.96, 0.87])
    templateblock.title(fig,"Shearer Speed through Cut Cycle",args)

    aql.ReportSave(plt, args)
 

The first line ensures that the report uses the tight-layout layout mechanism in MatPlotLib, which usually gives you a visually pleasing layout of all of the components on the page.

The call to templateblock.title adds a title block to each report. This is an optional step, but makes your reports look consistent and professional. You can make changes to your report template by editing the templateblock.py file in the reports directory.

The final call is to aql.ReportSave, which saves a PDF and PNG copy of your report - including the report name and date range - to the report directory.

Open the ARDI web interface, choose Reports from the sidebar and open your new report.

If you ever need to debug a report, you'll find a log.txt file in the folder where the reports Python file is located.

This file contains the compete command-line used to launch your report. You can copy this and paste it into a Command Prompt window to see the report being generated live and see any diagnostic info or exceptions.

To re-generate the saved report, delete the last.txt file.