Raspberry Temperature Humidity Data logger

I recently built a temperature and humidity data logger using a Raspberry Pi 3 B+ and a DHT22 sensor. My professional data logger stopped working so I decided to build my own to meet my lab needs. Some of the optics on my lab bench are worth the extra effort to keep humidity low. Low humidity avoids fungal deterioration over time on lens coatings. Here I present a DIY project to create a data logger so others may build one too.

Hardware

Software

The animation rolls the data so the last ~12 hours of data are always shown. The variable “lstMax” controls how much data should roll. Note how the shape of the data changes once I start working in my lab around noon.

I found and corrected three types of error generated by the DHT22 sensor:

• Other developers have included a time-out test so a temperature / humidity result of none is managed.
• Occasionally humidity values go way beyond the 100% upper limit. I catch and report this error in code by looking for humidity values above 100%.
• Another error comes from the temperature value. In this case the temperature for one result is about 20 degrees Fahrenheit lower than the temperature result just before or after this result. I catch this type of error by taking the median from 10 results – then report this median value.

The code starts with a few references I used to learn how to create a DIY temperature humidity data logging platform from my Raspberry Pi. Then I import packages and define the hardware layout of my DHT22 sensor. Initialization is complete when I define the plotting environment along with lists for plotting data and smoothing data using the median of 10 results. The main program loop is managed using Matplotlib’s animation module through regular calls to function “animate”.

Near the end of my design process I switched from averaging a block of 10 sensor results to taking the median of these results. The benefit was that random very low “noise” temperature results were eliminated. An average would have still changed the final value enough to create a false dip in the temperature plot. I am impressed with the overall stability of the DHT22 sensor. But a few reasonable tests improve the accuracy of the overall stream of reported data being logged.

The data smoothing algorithm I created is a bit over complex. I wanted the data logging display to start displaying results quickly while still enforcing the value of statistics median. So I start with a median of 3 sensor test results and increment with each animation loop to more data collection per logged entry up to my target of 10 test results per reported log entry. In this way, the plot looks ok after about 20 seconds.

Data logging with time required a little extra care. Matplotlib supports many time and date formatting features so the x-axis can change over time without every tick value being assigned a value. Even though the values are formatted as text results. The variable “myFmt” controls x-axis formatting. I wanted the data logging to roll over a 12 to 24 hour period so management of time formatting was critical.

Areas I hope to improve in the future:

• Show the plot only after two data points are ready to plot. Currently the plot looks “stupid” until then – about the first 20 seconds in time. Maybe a simple “show current values” only until enough data has been collected for a meaningful chart.
• Add some intelligence to Y limits. Currently the limits for temperature and humidity are fixed. Different labs and different areas of the world could use different limits and degrees C vs degrees F.
• Add a web browser GUI front end so I can view results while away from my lab. I hope to the use the REMI library for this feature. Optics in my lab prefer low humidity. So it would be nice to know what is happening while I am away. (Or maybe an email notice would be good too).
• Add storage of data to either a database or just to a csv file.
• Package the hardware off breadboard so it can remain reliable for longer periods of time or be used in lab experiments such as for the incubator I am building.
• Add a second DHT22 sensor to aid in calibration or in long term failure analysis.

Source Code

'''
Matplotlib animation of humidity and temperature 
        using DHT22 sensor on Raspberry Pi 3 B+
        Raspbian OS, python3, and Adafruit_DHT
        
Developer: BiophysicsLab.com

Last updated: June 24, 2020


References: 
        https://learn.sparkfun.com/tutorials/graph-sensor-data-with-python-and-matplotlib/update-a-graph-in-real-time
		
        https://pimylifeup.com/raspberry-pi-humidity-sensor-dht22/

        https://www.digikey.com/en/maker/projects/graph-sensor-data-with-python-and-matplotlib/93f919fd4e134c48bc5cd2bc0e5a5ba2
'''

import datetime as dt
import matplotlib.pyplot as plt
import matplotlib.animation as animation
import matplotlib.dates as mdates
from statistics import median
import Adafruit_DHT

DHT_SENSOR = Adafruit_DHT.DHT22
DHT_PIN = 4

print("Launch temperature and humidity Matplotlib animation using DHT22 sensor...")

# Create figure for plotting.
fig, (ax1, ax2) = plt.subplots(2,1)
myFmt = mdates.DateFormatter('%H:%M:%S')

# Create data to plot lists for animation.
lstMax = 100*24 # about 12 hours of data
xs = []
yst = []
ysh = []

# Manage averaging of data per list entry.
# Increase speed of initial results displayed: start with aveMin, 
# then incrementing up to target aveMax entries.
aveLst = {'aveMax': 10, 'aveMin': 3, 'ave': 0, 'ayst': [], 'aysh': []}

# This function is called periodically from FuncAnimation.
def animate(i, lstMax, xs, yst, ysh, aveLst):

    # Read temperature and Humidity from DHT22 sensor.
    humidity, temperature = Adafruit_DHT.read_retry(DHT_SENSOR, DHT_PIN)
    if humidity is not None and temperature is not None:
        temperature = round(temperature * 9/5.0 + 32,1)
        humidity = round(humidity,1)
        
        # Test for and ignore occasional erroneous high humidity values.
        # example: 52.5 for temperature, 3305.7 for humidity
        if humidity > 100.:
                print("Bad data skipped:", \
                dt.datetime.now().strftime('%H:%M:%S'), \
                str(temperature),str(humidity))
                return
                
        if aveLst['ave']< aveLst['aveMin']:
                print("Current:", \
                dt.datetime.now().strftime('%H:%M:%S'), \
                str(temperature),str(humidity))
                # Accumulate data without ploting.
                aveLst['ayst'].append(temperature)
                aveLst['aysh'].append(humidity)
                aveLst['ave'] += 1
                return
        else:
                # Add current time to xs, and median for yst and ysh, 
                # to animate plot.
                # Use median to catch occasional erroneous low 
                # temperature readings (ie low by approx. 20 deg F).
                xs.append(dt.datetime.now())
                yst.append(median(aveLst['ayst']))
                ysh.append(median(aveLst['aysh']))
                # Reset statistics lists.
                aveLst['ave'] = 0
                aveLst['ayst'].clear()
                aveLst['aysh'].clear()
                # Start display quickly then increase number of entries 
                # to use in median calculation up to aveMax.
                aveLst['aveMin'] = min(aveLst['aveMin']+1,aveLst['aveMax'])
                print("Dataset to average:", str(aveLst['aveMin']))


        # Limit x and y lists to lstMax items.
        xs = xs[-lstMax:]
        yst = yst[-lstMax:]
        ysh = ysh[-lstMax:]

        # Draw x and y lists.
        ax1.clear()
        ax1.plot(xs, yst,  color="red")
        ax2.clear()
        ax2.plot(xs, ysh, color="blue")

        # Format plot.
        plt.xticks(rotation=45, ha='right')
        # Leave a little room at the bottom for rotated xticks.
        plt.subplots_adjust(bottom=0.15)
        ax1.set_title('DHT22 Sensor Temperature & Humidity over Time')
        ax1.set_ylabel('Temperature (deg F)')
        ax1.set_ylim(65,80)
        ax1.grid()
        ax1.get_xaxis().set_ticklabels([])
        ax2.set_ylabel('Humidity (%RH)')
        ax2.set_ylim(45,65)
        ax2.xaxis.set_major_formatter(myFmt)
        ax2.grid()


# Set up plot to call animate() function periodically.
ani = animation.FuncAnimation(fig, animate, fargs=(lstMax, xs, yst, ysh, aveLst), interval=1000)
plt.show()