wview Weather Station On The Raspberry Pi 2
March 29, 2015 Leave a comment
Wview is weather station software used to collect, store, process, and publish weather data from a hardware based weather station. Wview is open source and Unix based so there are versions for Linux, BSD, Mac OSX, and more.
The NetCodger has been running wview on an old Linux PC for about three years, collecting data from a Davis Vantage Vue weather station. Wview has proven to be very powerful, reliable, and very customizable. Though customization does take a lot of work.
In the course of testing the newly released Raspberry Pi 2, the NetCodger sought to move his present wview installation over to a Raspberry Pi 2 to see if the Pi was up to the task. Not to mention free up some office space and reduce the electric bill.As it turns out the Raspberry Pi 2 can run wview with ease. CPU and memory use were minimal on the Raspberry Pi 2 while running wview.
Wview, like so many other software packages, assumes that it will be run with an Apache 2 webserver. The NetCodger is confident that the Raspberry Pi 2 could manage wview and Apache 2. But, he feels that the memory use and overall “weight” of Apache 2 is just unwarranted on a resource constrained device like the Raspberry Pi and instead opted to use the Lighttpd webserver to replace Apache 2.
Unrealeted to the webserver, there were performance issues on the Raspberry Pi 2 with a feature of wview that regenerates Hi/Low data records at startup if the Hi/Low database is missing or corrupted. Hi/Low regeneration can take a long time and the wview documentation warns of this. This is true of any system, but on the Pi it was taking more than 30 seconds per week’s worth of records. This meant that to regenerate from a three year data set of over 200,000 records it took well over an hour on the Raspberry Pi 2. That’s a very long startup. The need for such regeneration shouldn’t happen often, if ever, but it has been known to happen and this lengthy process is a cause for codgerly concern.
In an effort to reduce disk writes on the Raspberry Pi’s micro SD flash card and in hopes of speeding up wview’s Hi/Low regeneration, the NetCodger implemented a RAM disk and set up wview to store the Hi/Low Sqlite3 database(wview-hilow.sdb) and some other frequently changed files there. An init script to save the data to persistent storage on shutdown and load data from persistent storage on startup was also implemented. This significantly reduces the disk writes to the micro SD card, but regeneration performance was not improved. This perplexes the NetCodger.
Overall, wview is a fine choice for weather station software and wview runs very well on the Raspberry Pi 2. If you’d like to try it for yourself, the NetCodger has provided the following Bash script to quickly update a new Rasbian OS install, install wview, and create the RAM disk and startup environment for wview. There’s also a second optional script that can be used to import data from an existing wview installation, if you choose to retire your existing system. To use it, save it in a file on your Raspberry Pi. Make sure to make the file executable with the sudo chmod +x installWview.sh command and then run the file with sudo ./installWview.sh
installWview.sh
#!/bin/bash # installWview.sh # # Script to install WView weather station software on new Rapsberry Pi. # This script is not meant to be used on an already configured system. # This script could overwrite or break an existing confgiuration. # # By NetCodger 3/27/2015 # # Make sure this script only runs on a Raspberry Pi. if ! uname -a | grep "raspberrypi"; then echo "This script is meant to only run on a Raspberry Pi." echo "This does not appear to be a Raspberry Pi." read -n 1 -p "Pres any key exit" exit -1 fi # Make sure that we are root if [ $(id -u) != 0 ]; then echo "Insufficient privilege to execute this script." echo echo "Please re-run the script with sudo installWView.sh" read -n 1 -p "Pres any key exit" exit -1 fi # Are you sure? echo "This script is for installing WView on a new Raspberry Pi installatoin." echo "Installing on an already configured system could destroy any previous modifications." read -p "Are you sure you want to continue? (y/n)" -n 1 -r echo # (optional) move to a new line if [[ ! $REPLY =~ ^[Yy]$ ]]; then exit 1 fi # Update /etc/apt/sources.list to include download location for Wview software, if it does not already exist. if ! grep -Fxq "deb http://www.wviewweather.com/apt/wheezy wheezy main" /etc/apt/sources.list; then echo "deb http://www.wviewweather.com/apt/wheezy wheezy main" >> /etc/apt/sources.list echo "deb-src http://www.wviewweather.com/apt/wheezy wheezy main" >> /etc/apt/sources.list fi # Update installed Raspbian(Debian) software. apt-get update apt-get --assume-yes --force-yes upgrade # Update Pi firmware. rpi-update # Prompt user to set timezone as necessary. dpkg-reconfigure tzdata # Install web server and PHP apt-get --no-install-recommends --assume-yes --force-yes --fix-missing install lighttpd apt-get --no-install-recommends --assume-yes --force-yes --fix-missing install php5 php5-sqlite php5-cgi # Configure lighttpd to enable PHP if ! grep -Fxq '.php" => "/usr/bin/php5-cgi' /etc/lighttpd/conf-enabled/10-cgi-php.conf; then echo 'server.modules += ("mod_cgi")' >> /etc/lighttpd/conf-enabled/10-cgi-php.conf echo ' cgi.assign = (".php" => "/usr/bin/php5-cgi")' >> /etc/lighttpd/conf-enabled/10-cgi-php.conf fi lighttpd-enable-mod fastcgi fastcgi-php # Install WView weather station software.(But without installing Apache2.) apt-get --no-install-recommends --assume-yes --force-yes --fix-missing install wview apache2- # Create simlinks manually, since WView only does it for Apache2. cd /var/www ln -s /var/lib/wview/img weather ln -s /var/lib/wviewmgmt wviewmgmt cd /var/lib/wviewmgmt ln -s system_status.php index.php # Adjust fille permissions. # Make wview config files writable for http user. chmod 777 /etc/wview chmod 666 /etc/wview/wview-conf.sdb chmod 666 /etc/wview/wview-binary # Create a ramdisk for Hi Low data. # Saves SD card write cycles and # speeds up Hi Low regeneration. if ! grep -Fxq "tmpfs /wviewtmp tmpfs defaults,mode=0755,size=30m 0 0" /etc/fstab; then mkdir /wviewtmp echo "tmpfs /wviewtmp tmpfs defaults,mode=0755,size=30m 0 0" >> /etc/fstab mount -a fi # Create persistent data store. Needed due to ramdisk. mv /var/lib/wview/archive /var/lib/wview/archive-persistent mv /var/lib/wview/img /var/lib/wview/img-persistent # Link database files so that WView can find them. if [ ! -d /var/lib/wview/archive ]; then mkdir /var/lib/wview/archive cat > /var/lib/wview/archive/README-IMPORTANT << 'EOT1' Due to the use of a ramdisk for the /wviewtmp directory this directory should contain the following soft links. ln -s /var/lib/wview/archive-persistent/wview-archive.sdb /var/lib/wview/archive/wview-archive.sdb ln -s /var/lib/wview/archive-persistent/wview-archive.sql /var/lib/wview/archive/wview-archive.sql ln -s /var/lib/wview/archive-persistent/wview-history.sdb /var/lib/wview/archive/wview-history.sdb ln -s /var/lib/wview/archive-persistent/wview-noaa.sdb /var/lib/wview/archive/wview-noaa.sdb ln -s /wviewtmp/wview-hilow.sdb /var/lib/wview/archive/wview-hilow.sdb EOT1 ln -s /var/lib/wview/archive-persistent/wview-archive.sdb /var/lib/wview/archive/wview-archive.sdb ln -s /var/lib/wview/archive-persistent/wview-archive.sql /var/lib/wview/archive/wview-archive.sql ln -s /var/lib/wview/archive-persistent/wview-history.sdb /var/lib/wview/archive/wview-history.sdb ln -s /var/lib/wview/archive-persistent/wview-noaa.sdb /var/lib/wview/archive/wview-noaa.sdb ln -s /wviewtmp/wview-hilow.sdb /var/lib/wview/archive/wview-hilow.sdb ln -s /wviewtmp/img /var/lib/wview/img fi # Create a service script that copies wview-hilow.sdb on startup or shutdown. cat > /etc/init.d/tmpfsToPersistent << 'EOT2' #! /bin/bash # /etc/init.d/tmpfsToPersistent ### BEGIN INIT INFO # Provides: tmpfsToPersistent # Required-Start: $local_fs $network $time $syslog ntp # Required-Stop: $local_fs $network $time $syslog ntp # Default-Start: 2 3 4 5 # Default-Stop: 0 1 6 # Short-Description: Script to save data at shutdown. # Description: Copies wview database from persistent to tmpfs at start and vice versa at shutdown. ### END INIT INFO case "$1" in start) echo "Starting copy /var/lib/wview/archive-persistent/wview-hilow.sdb to /wviewtmp/wview-hilow.sdb a tmpfs location." if mount | grep "tmpfs on /wviewtmp" > /dev/null; then cp -a /var/lib/wview/archive-persistent/wview-hilow.sdb /wviewtmp/ cp -ar /var/lib/wview/img-persistent/ /wviewtmp/img/ else echo "/wview not mounted on tmpfs" fi ;; stop) echo "Stopping copying /wviewtmp/wview-hilow.sdb to persistent storage" cp -a /wviewtmp/wview-hilow.sdb /var/lib/wview/archive-persistent/ cp -ar /wviewtmp/img/* /var/lib/wview/img-persistent/ ;; *) echo "Usage: /etc/init.d/tmpfsToPersistent {start|stop}" exit 1 ;; esac exit 0 EOT2 chmod 755 /etc/init.d/tmpfsToPersistent update-rc.d tmpfsToPersistent defaults 10 50 # Setup automatic WView service start/stop sed -i 's#$local_fs $network $time $syslog#$local_fs $network $time $syslog tmpfsToPersistent#' /etc/init.d/wview update-rc.d wview defaults 90 10 # Reboot and begin configuration of WView. echo echo "----------------------------------" echo "Reboot Raspberry Pi and begin using Wview?" echo "See WView manual about wviewconfig." echo echo "Choose NO if you plan to run the RestoreWview script to import prior data." echo read -p "Press y to reboot and n to exit" -n 1 -r echo # (optional) move to a new line if [[ $REPLY =~ ^[Yy]$ ]]; then reboot fi
RestoreWview.sh
#!/bin/bash # RestoreWview.sh # # Script to import WView weather station databases from Sqlite3 dumps. # It is intended to facilitate moving Wview data to a new system. # It is not a complete backup/restore utility. # # By NetCodger 3/27/2015 # # # Important Note! Paths are for Debain based Raspbian distributions. # Verify and adjust desination path for new system if necessary. # ArchivePath="/var/lib/wview/archive-persistent" ConfPath="/etc/wview" # Dump databases on source system with # sudo sqlite3 /var/lib/wview/archive/wview-archive.sdb .dump > wview-archive.sdb.sql # sudo sqlite3 /var/lib/wview/archive/wview-hilow.sdb .dump > wview-hilow.sdb.sql # sudo sqlite3 /var/lib/wview/archive/wview-history.sdb .dump > wview-history.sdb.sql # sudo sqlite3 /etc/wview/wview-conf.sdb .dump > wview-conf.sdb.sql # Then copy dumped .sql files to target system. # Also copy /etc/wview/wview-binary from source to destination system. # Make sure that we are root if [ $(id -u) != 0 ]; then echo "Insufficient privilege to execute this script." echo echo "Please re-run the script with sudo RestoreWView.sh" read -n 1 -p "Pres any key exit" exit -1 fi # Are you sure? echo "This script is for restoring WView databases onto a new installatoin." echo "Installing on an existing Wview system will clobber any previous modifications." read -p "Are you sure you want to continue? (y/n)" -n 1 -r echo # (optional) move to a new line if [[ ! $REPLY =~ ^[Yy]$ ]]; then exit 1 fi # Stop WView. /etc/init.d/wview stop # Remove existing files echo "Deleting old database files." rm $ArchivePath/wview-archive.sdb rm $ArchivePath/wview-hilow.sdb rm $ArchivePath/wview-history.sdb rm $ConfPath/wview-conf.sdb echo "Importing data dumps." sqlite3 "$ArchivePath/wview-archive.sdb" < ./wview-archive.sdb.sql sqlite3 "$ArchivePath/wview-hilow.sdb" < ./wview-hilow.sdb.sql sqlite3 "$ArchivePath/wview-history.sdb" < ./wview-history.sdb.sql sqlite3 "$ConfPath/wview-conf.sdb" < ./wview-conf.sdb.sql cp ./wview-binary /etc/wview/ # Run wviewconfig to import config database. echo "Running wvconfig. Press Enter to accept previous settings(default)." wviewconfig get > $PWD/tempconf.txt wviewconfig set $PWD/tempconf.txt rm $PWD/tempconf.txt # Flush and import files stored on tmpfs rm -rf /wviewtmp/* cp -a /var/lib/wview/archive-persistent/wview-hilow.sdb /wviewtmp/ cp -ar /var/lib/wview/img-persistent/ /wviewtmp/img/ # Start WView. echo echo "WView is ready to start." echo "If Hi Low data must be regenerated, it can take a long time to start." echo "If the archive database is large, it can take a VERY long time." echo "Tail /var/log/messages, to see if HiLow is regenerating." read -p "Do you wish to start WView? (y/n)" -n 1 -r echo # (optional) move to a new line if [[ ! $REPLY =~ ^[Yy]$ ]]; then exit 1 else /etc/init.d/wview start fi # Hi/Lo regeneration is painfully slow at 38 seconds per week on a Raspberry Pi 2.