Merge pull request #1201 from pi-hole/development

[RELEASE] Pi-hole Core 2.12.1

.codeclimate.yml

@@ -1,3 +0,0 @@
-engines:
-  shellcheck:
-    enabled: true

.github/ISSUE_TEMPLATE.md

@@ -6,21 +6,12 @@
 
 **How familiar are you with the codebase?:**
 
-- [] 1 (very unfamiliar)
-- [] 2
-- [] 3
-- [] 4
-- [] 5
-- [] 6
-- [] 7
-- [] 8
-- [] 9
-- [] 10 (very familiar)
+_{replace this text with a number from 1 to 10, with 1 being not familiar, and 10 being very familiar}_
 
 ---
 **[FEATURE REQUEST | QUESTION | OTHER]:**
 
-_{replace this section with your content or delete if not a FEATURE REQUEST/QUESTION/OTHER}_
+Please [submit your feature request here](https://discourse.pi-hole.net/c/feature-requests), so it is votable by the community.  It's also easier for us to track.
 
 **[BUG | ISSUE] Expected Behaviour:**
 
@@ -35,8 +26,8 @@ _{replace this section with your content or delete if not a FEATURE REQUEST/QUES
 -
 -
 
-**(Optional) Debug Log generated by `pihole -d`:**
+**(Optional) Debug token generated by `pihole -d`:**
 
-`http://termbin.com/<something>`
+`<token>`
 
 _This template was created based on the work of [`udemy-dl`](https://github.com/nishad/udemy-dl/blob/master/LICENSE)._

.github/PULL_REQUEST_TEMPLATE.md

@@ -1,4 +1,6 @@
-**By submitting this pull request, I confirm the following (please check boxes, eg [X])Failure to fill the template will close your PR:**
+**By submitting this pull request, I confirm the following (please check boxes, eg [X]) _Failure to fill the template will close your PR_:**
+
+***Please submit all pull requests against the `development` branch. Failure to do so will delay or deny your request***
 
 - [] I have read and understood the [contributors guide](https://github.com/pi-hole/pi-hole/blob/master/CONTRIBUTING.md).
 - [] I have checked that [another pull request](https://github.com/pi-hole/pi-hole/pulls) for this purpose does not exist.
@@ -8,16 +10,7 @@
 
 **How familiar are you with the codebase?:**
 
-- [] 1 (very unfamiliar)
-- [] 2
-- [] 3
-- [] 4
-- [] 5
-- [] 6
-- [] 7
-- [] 8
-- [] 9
-- [] 10 (very familiar)
+_{replace this text with a number from 1 to 10, with 1 being not familiar, and 10 being very familiar}_
 
 ---
 _{replace this line with your pull request content}_

.gitignore

@@ -1 +1,7 @@
 .DS_Store
+*.pyc
+*.swp
+__pycache__
+.cache
+.pullapprove.yml
+

.idea/codeStyleSettings.xml

@@ -0,0 +1,25 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<project version="4">
+  <component name="ProjectCodeStyleSettingsManager">
+    <option name="PER_PROJECT_SETTINGS">
+      <value>
+        <option name="OTHER_INDENT_OPTIONS">
+          <value>
+            <option name="INDENT_SIZE" value="2" />
+            <option name="CONTINUATION_INDENT_SIZE" value="8" />
+            <option name="TAB_SIZE" value="2" />
+            <option name="USE_TAB_CHARACTER" value="false" />
+            <option name="SMART_TABS" value="false" />
+            <option name="LABEL_INDENT_SIZE" value="0" />
+            <option name="LABEL_INDENT_ABSOLUTE" value="false" />
+            <option name="USE_RELATIVE_INDENTS" value="false" />
+          </value>
+        </option>
+        <MarkdownNavigatorCodeStyleSettings>
+          <option name="RIGHT_MARGIN" value="72" />
+        </MarkdownNavigatorCodeStyleSettings>
+      </value>
+    </option>
+    <option name="USE_PER_PROJECT_SETTINGS" value="true" />
+  </component>
+</project>

.pullapprove.yml

@@ -25,7 +25,7 @@ groups:
         - development        
     required: 2
     teams:
-      - gravity
+      - approvers
   
   master:
     approve_by_comment:
@@ -33,6 +33,6 @@ groups:
     conditions:
       branches:
         - master
-    required: 3
+    required: -1
     teams:
-      - gravity
+      - admin

.travis.yml

@@ -0,0 +1,10 @@
+sudo: required
+services:
+  - docker
+language: python
+python:
+  - "2.7"
+install:
+  - pip install -r requirements.txt
+
+script: py.test -vv

CONTRIBUTING.md

@@ -33,5 +33,6 @@ When requesting or submitting new features, first consider whether it might be u
 
 - Submit Pull Requests to the **development branch only**.
 - Before Submitting your Pull Request, merge `development` with your new branch and fix any conflicts. (Make sure you don't break anything in development!)
+- Please use the [Google Style Guide for Shell](https://google.github.io/styleguide/shell.xml) for your code submission styles. 
 - Commit Unix line endings.
 - (Optional fun) keep to the theme of Star Trek/black holes/gravity.

README.md

@@ -1,52 +1,74 @@
-[![Bountysource](https://www.bountysource.com/badge/tracker?tracker_id=3011939)](https://www.bountysource.com/trackers/3011939-pi-hole-pi-hole?utm_source=3011939&utm_medium=shield&utm_campaign=TRACKER_BADGE) [![Code Climate](https://codeclimate.com/github/pi-hole/pi-hole/badges/gpa.svg)](https://codeclimate.com/github/pi-hole/pi-hole)
+<p align="center">
+<a href=https://www.bountysource.com/trackers/3011939-pi-hole-pi-hole?utm_source=3011939&utm_medium=shield&utm_campaign=TRACKER_BADGE><img src="https://www.bountysource.com/badge/tracker?tracker_id=3011939"></a>
+<a href="https://www.codacy.com/app/Pi-hole/pi-hole?utm_source=github.com&amp;utm_medium=referral&amp;utm_content=pi-hole/pi-hole&amp;utm_campaign=Badge_Grade"><img src="https://api.codacy.com/project/badge/Grade/c558a0f8d7124c99b02b84f0f5564238"/></a>
+<a href=https://travis-ci.org/pi-hole/pi-hole><img src="https://travis-ci.org/pi-hole/pi-hole.svg?branch=development"></a>
+</p>
 
-# Automated Install
+<p align="center">
+<a href=https://discourse.pi-hole.net><img src="https://assets.pi-hole.net/static/Vortex_text.png" width=210></a>
+</p>
 
-Designed For Raspberry Pi A+, B, B+, 2, 3B, and Zero (with an Ethernet/Wi-Fi adapter)
-Works on most Debian and CentOS/RHEL based distributions!
+## The multi-platform, network-wide ad blocker
 
-1.  Install Raspbian
-2.  Run the command below (downloads [this script](https://github.com/pi-hole/pi-hole/blob/master/automated%20install/basic-install.sh) in case you want to read over it first!)
+Block ads for **all** your devices _without_ the need to install client-side software.  The Pi-hole blocks ads at the DNS-level, so all your devices are protected.
 
-## `curl -L https://install.pi-hole.net | bash`
+- Web Browsers
+- Cell Phones
+- Smart TV's
+- Internet-connected home automation
+- Anything that communicates with the Internet
 
-### Alternative Semi-Automated install
+<p align="center">
+<a href=http://www.digitalocean.com/?refcode=344d234950e1><img src="https://assets.pi-hole.net/static/DOHostingSlug.png"></a>
+</p>
+
+## Your Support Still Matters
+
+Digital Ocean helps with our infrastructure, but our developers are all volunteers so *your donations help keep us innovating*. Sending a donation using our links below helps us offset a portion of our monthly costs.
+
+-   ![Paypal](https://assets.pi-hole.net/static/paypal.png) [Donate via PayPal](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY)
+-   ![Bitcoin](https://assets.pi-hole.net/static/Bitcoin.png) Bitcoin Address: 1GKnevUnVaQM2pQieMyeHkpr8DXfkpfAtL
+
+### One-Step Automated Install
+1.  Install a [supported operating system](https://discourse.pi-hole.net/t/hardware-software-requirements/273/1)
+2.  Run the command below (it downloads [this script](https://github.com/pi-hole/pi-hole/blob/master/automated%20install/basic-install.sh) in case you want to read over it first!)
+
+### `curl -sSL https://install.pi-hole.net | bash`
+
+#### Alternative Semi-Automated Install Methods
+_If you wish to read over the script before running it, run `nano basic-install.sh` to open the file in a text viewer._
+
+##### Clone our repository and run the automated installer from your device.
+
+```
+git clone --depth 1 https://github.com/pi-hole/pi-hole.git Pi-hole
+cd Pi-hole/automated\ install/
+bash basic-install.sh
+```
+
+##### Or
 
 ```bash
 wget -O basic-install.sh https://install.pi-hole.net
-chmod +x basic-install.sh
-./basic-install.sh
+bash basic-install.sh
 ```
 
-If you wish to read over the script before running it, then after the `wget` command, do `nano basic-install.sh` to open a text viewer
+Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](https://discourse.pi-hole.net/t/how-do-i-configure-my-devices-to-use-pi-hole-as-their-dns-server/245) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to [use the Raspberry Pi as its DNS server](http://pi-hole.net/faq/how-do-i-use-the-pi-hole-as-my-dns-server/).
 
-Once installed, [configure your router to have **DHCP clients use the Pi as their DNS server**](http://pi-hole.net/faq/can-i-set-the-pi-hole-to-be-the-dns-server-at-my-router-so-i-dont-have-to-change-settings-for-my-devices/) and then any device that connects to your network will have ads blocked without any further configuration. Alternatively, you can manually set each device to [use the Raspberry Pi as its DNS server](http://pi-hole.net/faq/how-do-i-use-the-pi-hole-as-my-dns-server/).
+## What is Pi-hole and how do I install it?
+<p align="center">
+<a href=https://www.youtube.com/watch?v=vKWjx1AQYgs><img src="https://assets.pi-hole.net/static/global.png"></a>
+</p>
 
-## How To Install Pi-hole
-
-[![60-second install tutorial](http://i.imgur.com/5TEc3a6.png)](https://www.youtube.com/watch?v=TzFLJqUeirA)
-
-## How It Works
-
-**Watch the 60-second video below to get a quick overview**
-
-[![Pi-hole exlplained](http://i.imgur.com/pG1m937.png)](https://youtu.be/9Eti3xibiho)
-
-## Pi-hole Is Free, But Powered By Your Donations
-
-Send a one-time donation or sign up for Optimal.com's service using our link below to provide us with a small portion of the monthly fee.
-
--   ![Paypal](http://i.imgur.com/3muNfxu.png) : [Donate](https://www.paypal.com/cgi-bin/webscr?cmd=_s-xclick&hosted_button_id=3J2L3Z4DHW9UY)
--   ![Flattr](http://i.imgur.com/ZFceFRu.png) : [Donate](https://flattr.com/submit/auto?user_id=jacobsalmela&url=https://github.com/pi-hole/pi-hole)
--   ![Bitcoin](http://i.imgur.com/FIlmOMG.png) : 1GKnevUnVaQM2pQieMyeHkpr8DXfkpfAtL
 
 ## Get Help Or Connect With Us On The Web
 
--   [@The_Pi_Hole](https://twitter.com/The_Pi_Hole)
--   [/r/pihole](https://www.reddit.com/r/pihole/)
--   [Pi-hole YouTube channel](https://www.youtube.com/channel/UCT5kq9w0wSjogzJb81C9U0w)
--   [Wiki](https://github.com/pi-hole/pi-hole/wiki/Customization)
--   [FAQs](https://pi-hole.net/help/)
+-   [Users Forum](https://discourse.pi-hole.net/)
+-   [FAQs](https://discourse.pi-hole.net/c/faqs)
+-   [Wiki](https://github.com/pi-hole/pi-hole/wiki)
+-   ![Twitter](https://assets.pi-hole.net/static/twitter.png) [Tweet @The_Pi_Hole](https://twitter.com/The_Pi_Hole)
+-   ![Reddit](https://assets.pi-hole.net/static/reddit.png) [Reddit /r/pihole](https://www.reddit.com/r/pihole/)
+-   ![YouTube](https://assets.pi-hole.net/static/youtube.png)  [Pi-hole channel](https://www.youtube.com/channel/UCT5kq9w0wSjogzJb81C9U0w)
 -   [![Join the chat at https://gitter.im/pi-hole/pi-hole](https://badges.gitter.im/pi-hole/pi-hole.svg)](https://gitter.im/pi-hole/pi-hole?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge)
 
 ## Technical Details
@@ -55,25 +77,34 @@ The Pi-hole is an **advertising-aware DNS/Web server**. If an ad domain is queri
 
 ### Gravity
 
-The [gravity.sh](https://github.com/pi-hole/pi-hole/blob/master/gravity.sh) does most of the magic. The script pulls in ad domains from many sources and compiles them into a single list of [over 1.6 million entries](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0) (if you decide to use the [mahakala list](https://github.com/pi-hole/pi-hole/commit/963eacfe0537a7abddf30441c754c67ca1e40965)).
+The [gravity.sh](https://github.com/pi-hole/pi-hole/blob/master/gravity.sh) does most of the magic. The script pulls in ad domains from many sources and compiles them into a single list of [over 1.6 million entries](http://jacobsalmela.com/block-millions-ads-network-wide-with-a-raspberry-pi-hole-2-0) (if you decide to use the [mahakala list](https://github.com/pi-hole/pi-hole/commit/963eacfe0537a7abddf30441c754c67ca1e40965)). This script is controlled by the `pihole` command. Please run `pihole -h` to see what commands can be run via `pihole`.
 
-## Web Interface
 
-The [Web interface](https://github.com/jacobsalmela/AdminLTE#pi-hole-admin-dashboard) will be installed automatically so you can view stats and change settings. You can find it at:
+
+#### Other Operating Systems
+
+The automated install is only for a clean install of a Debian family or Fedora based system, such as the Raspberry Pi. However, this script will work for most UNIX-like systems, some with some slight **modifications** that we can help you work through. If you can install `dnsmasq` and a Webserver, it should work OK. If there are other platforms you'd like supported, let us know.
+
+### Web Interface
+
+The [Web interface](https://github.com/pi-hole/AdminLTE#pi-hole-admin-dashboard) will be installed automatically so you can view stats and change settings. You can find it at:
 
 `http://192.168.1.x/admin/index.php` or `http://pi.hole/admin`
 
-![Pi-hole Advanced Stats Dashboard](http://i.imgur.com/gTq2GbS.png)
+![Pi-hole Advanced Stats Dashboard](https://assets.pi-hole.net/static/dashboard.png)
 
 ### Whitelist and blacklist
 
-Domains can be whitelisted and blacklisted using two pre-installed scripts. See [the wiki page](https://github.com/pi-hole/pi-hole/wiki/Whitelisting-and-Blacklisting) for more details ![Whitelist editor in the Web interface](http://i.imgur.com/ogu2ewg.png)
+Domains can be whitelisted and blacklisted using either the web interface or the command line. See [the wiki page](https://github.com/pi-hole/pi-hole/wiki/Whitelisting-and-Blacklisting) for more details
+<p align="center">
+<a href=https://github.com/pi-hole/pi-hole/wiki/Whitelisting-and-Blacklisting><img src="https://assets.pi-hole.net/static/controlpanel.png"></a>
+</p>
 
 ## API
 
 A basic read-only API can be accessed at `/admin/api.php`. It returns the following JSON:
 
-```json
+``` json
 {
     "domains_being_blocked": "136708",
     "dns_queries_today": "18108",
@@ -91,7 +122,7 @@ You can view [real-time stats](http://pi-hole.net/faq/install-the-real-time-lcd-
 ## Pi-hole Projects
 
 -   [Pi-hole stats in your Mac's menu bar](https://getbitbar.com/plugins/Network/pi-hole.1m.py)
--   [Get LED alerts for each blocked ad](http://www.stinebaugh.info/get-led-alerts-for-each-blocked-ad-using-pi-hole/)
+-   [Get LED alerts for each blocked ad](http://thetimmy.silvernight.org/pages/endisbutton/)
 -   [Pi-hole on Ubuntu 14.04 on VirtualBox](http://hbalagtas.blogspot.com/2016/02/adblocking-with-pi-hole-and-ubuntu-1404.html)
 -   [Docker Pi-hole container (x86 and ARM)](https://hub.docker.com/r/diginc/pi-hole/)
 -   [Splunk: Pi-hole Visualizser](https://splunkbase.splunk.com/app/3023/)
@@ -103,6 +134,8 @@ You can view [real-time stats](http://pi-hole.net/faq/install-the-real-time-lcd-
 -   [Pi-hole on/off button](http://thetimmy.silvernight.org/pages/endisbutton/)
 -   [Minibian Pi-hole](http://munkjensen.net/wiki/index.php/See_my_Pi-Hole#Minibian_Pi-hole)
 -   [Windows Tray Stat Application](https://github.com/goldbattle/copernicus)
+-   [Let your blink1 device blink when Pi-hole filters ads](https://gist.github.com/elpatron68/ec0b4c582e5abf604885ac1e068d233f)
+-   [Pi-Hole Prometheus exporter](https://github.com/nlamirault/pihole_exporter) : a [Prometheus](https://prometheus.io/) exporter for Pi-Hole
 
 ## Coverage
 
@@ -119,7 +152,4 @@ You can view [real-time stats](http://pi-hole.net/faq/install-the-real-time-lcd-
 -   [Foolish Tech Show](https://youtu.be/bYyena0I9yc?t=2m4s)
 -   [Pi-hole on Ubuntu](http://www.boyter.org/2015/12/pi-hole-ubuntu-14-04/)
 -   [Catchpoint: iOS 9 Ad Blocking](http://blog.catchpoint.com/2015/09/14/ad-blocking-apple/)
-
-## Other Operating Systems
-
-This script will work for other UNIX-like systems with some slight **modifications**. As long as you can install `dnsmasq` and a Webserver, it should work OK. The automated install is only for a clean install of a Debian based system, such as the Raspberry Pi.
+-   [Build an Ad-Blocker for less than 10$ with Orange-Pi](http://www.devacron.com/orangepi-zero-as-an-ad-block-server-with-pi-hole/)

adlists.default

@@ -1,4 +1,4 @@
-## Pi-hole ad-list default sources. Updated 22/05/2016 #########################
+## Pi-hole ad-list default sources. Updated 29/10/2016 #########################
 #                                                                              #
 #  To make changes to this file:                                               #
 #    1. run `cp /etc/pihole/adlists.default /etc/pihole/adlists.list`          #
@@ -28,7 +28,7 @@ https://hosts-file.net/ad_servers.txt
 #http://adblock.mahakala.is/
 
 # ADZHOSTS list. Has been known to block legitimate domains
-#http://optimate.dl.sourceforge.net/project/adzhosts/HOSTS.txt
+#http://pilotfiber.dl.sourceforge.net/project/adzhosts/HOSTS.txt
 
 # Windows 10 telemetry list
 #https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/win10/spy.txt
@@ -37,7 +37,7 @@ https://hosts-file.net/ad_servers.txt
 #http://securemecca.com/Downloads/hosts.txt
 
 # Quidsup's tracker list
-https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
+#https://raw.githubusercontent.com/quidsup/notrack/master/trackers.txt
 
 # Block the BBC News website Breaking News banner
 #https://raw.githubusercontent.com/BreakingTheNews/BreakingTheNews.github.io/master/hosts

advanced/01-pihole.conf

@@ -20,13 +20,8 @@
 #                        OR IN /etc/dnsmasq.conf                              #
 ###############################################################################
 
-address=/pi.hole/@IPv4@
-address=/pi.hole/@IPv6@
-
-address=/@HOSTNAME@/@IPv4@
-address=/@HOSTNAME@/@IPv6@
-
 addn-hosts=/etc/pihole/gravity.list
+addn-hosts=/etc/pihole/local.list
 
 domain-needed
 

advanced/Scripts/chronometer.sh

@@ -15,68 +15,23 @@
 piLog="/var/log/pihole.log"
 gravity="/etc/pihole/gravity.list"
 
-today=$(date "+%b %e")
+. /etc/pihole/setupVars.conf
 
-CalcBlockedDomains() {
-	CheckIPv6
-	if [ -e "${gravity}" ]; then
-		#Are we IPV6 or IPV4?
-		if [[ -n ${piholeIPv6} ]]; then
-			#We are IPV6
-			blockedDomainsTotal=$(wc -l /etc/pihole/gravity.list | awk '{print $1/2}')
-		else
-			#We are IPV4
-			blockedDomainsTotal=$(wc -l /etc/pihole/gravity.list | awk '{print $1}')
-		fi
-	else
-		blockedDomainsTotal="Err."
-	fi
-}
-
-CalcQueriesToday() {
-	if [ -e "${piLog}" ]; then
-		queriesToday=$(cat "${piLog}" | grep "${today}" | awk '/query/ {print $6}' | wc -l)
-	else
-		queriesToday="Err."
-	fi
-}
-
-CalcblockedToday() {
-	if [ -e "${piLog}" ] && [ -e "${gravity}" ];then
-		blockedToday=$(cat ${piLog} | awk '/\/etc\/pihole\/gravity.list/ && !/address/ {print $6}' | wc -l)
-	else
-		blockedToday="Err."
-	fi
-}
-
-CalcPercentBlockedToday() {
-	if [ "${queriesToday}" != "Err." ] && [ "${blockedToday}" != "Err." ]; then
-		if [ "${queriesToday}" != 0 ]; then #Fixes divide by zero error :)
-			#scale 2 rounds the number down, so we'll do scale 4 and then trim the last 2 zeros
-			percentBlockedToday=$(echo "scale=4; ${blockedToday}/${queriesToday}*100" | bc)
-			percentBlockedToday=$(sed 's/.\{2\}$//' <<< "${percentBlockedToday}")
-		else
-			percentBlockedToday=0
-		fi
-	fi
-}
-
-CheckIPv6() {
-	piholeIPv6file="/etc/pihole/.useIPv6"
-	if [[ -f ${piholeIPv6file} ]];then
-		# If the file exists, then the user previously chose to use IPv6 in the automated installer
-		piholeIPv6=$(ip -6 route get 2001:4860:4860::8888 | awk -F " " '{ for(i=1;i<=NF;i++) if ($i == "src") print $(i+1) }')
-	fi
+# Borrowed/modified from https://gist.github.com/cjus/1047794
+function GetJSONValue {
+    retVal=$(echo $1 | sed 's/\\\\\//\//g' | \
+                       sed 's/[{}]//g' | \
+                       awk -v k="text" '{n=split($0,a,","); for (i=1; i<=n; i++) print a[i]}' | \
+                       sed 's/\"\:/\|/g' | \
+                       sed 's/[\,]/ /g' | \
+                       sed 's/\"//g' | \
+                       grep -w $2)
+    echo ${retVal##*|}
 }
 
 outputJSON() {
-	CalcQueriesToday
-	CalcblockedToday
-	CalcPercentBlockedToday
-
-	CalcBlockedDomains
-
-	printf '{"domains_being_blocked":"%s","dns_queries_today":"%s","ads_blocked_today":"%s","ads_percentage_today":"%s"}\n' "$blockedDomainsTotal" "$queriesToday" "$blockedToday" "$percentBlockedToday"
+	json=$(curl -s -X GET http://127.0.0.1/admin/api.php?summaryRaw)
+	echo ${json}
 }
 
 normalChrono() {
@@ -88,7 +43,7 @@ normalChrono() {
 		echo "|  _/ |___| ' \/ _ \ / -_)"
 		echo "|_| |_|   |_||_\___/_\___|"
 		echo ""
-		echo "        $(ifconfig eth0 | awk '/inet addr/ {print $2}' | cut -d':' -f2)"
+		echo "        ${IPV4_ADDRESS}"
 		echo ""
 		uptime | cut -d' ' -f11-
 		#uptime -p	#Doesn't work on all versions of uptime
@@ -97,24 +52,17 @@ normalChrono() {
 		# Uncomment to continually read the log file and display the current domain being blocked
 		#tail -f /var/log/pihole.log | awk '/\/etc\/pihole\/gravity.list/ {if ($7 != "address" && $7 != "name" && $7 != "/etc/pihole/gravity.list") print $7; else;}'
 
-		#uncomment next 4 lines to use original query count calculation
-		#today=$(date "+%b %e")
-		#todaysQueryCount=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ {print $7}' | wc -l)
-		#todaysQueryCountV4=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ && /\[A\]/ {print $7}' | wc -l)
-		#todaysQueryCountV6=$(cat /var/log/pihole.log | grep "$today" | awk '/query/ && /\[AAAA\]/ {print $7}' | wc -l)
+		json=$(curl -s -X GET http://127.0.0.1/admin/api.php?summaryRaw)
 
+    domains=$(printf "%'.f" $(GetJSONValue ${json} "domains_being_blocked")) #add commas in
+    queries=$(printf "%'.f" $(GetJSONValue ${json} "dns_queries_today"))
+    blocked=$(printf "%'.f" $(GetJSONValue ${json} "ads_blocked_today"))
+    LC_NUMERIC=C percentage=$(printf "%0.2f\n" $(GetJSONValue ${json} "ads_percentage_today")) #2 decimal places
 
-		CalcQueriesToday
-		CalcblockedToday
-		CalcPercentBlockedToday
+		echo "Blocking:      ${domains}"
+		echo "Queries:       ${queries}"
 
-		CalcBlockedDomains
-
-		echo "Blocking:      ${blockedDomainsTotal}"
-		#below commented line does not add up to todaysQueryCount
-		#echo "Queries:       $todaysQueryCountV4 / $todaysQueryCountV6"
-		echo "Queries:       ${queriesToday}" #same total calculation as dashboard
-	  echo "Pi-holed:      ${blockedToday} (${percentBlockedToday}%)"
+		echo "Pi-holed:      ${blocked} (${percentage}%)"
 
 		sleep 5
 	done
@@ -131,7 +79,7 @@ displayHelp() {
 :::  -j, --json		output stats as JSON formatted string
 :::  -h, --help		display this help text
 EOM
-    exit 1
+    exit 0
 }
 
 if [[ $# = 0 ]]; then

advanced/Scripts/list.sh

@@ -15,6 +15,7 @@ basename=pihole
 piholeDir=/etc/${basename}
 whitelist=${piholeDir}/whitelist.txt
 blacklist=${piholeDir}/blacklist.txt
+readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
 reload=false
 addmode=true
 verbose=true
@@ -47,12 +48,25 @@ helpFunc() {
 :::  -h, --help				Show this help dialog
 :::  -l, --list				Display your ${word}listed domains
 EOM
-	exit 1
+if [[ "${letter}" == "b" ]]; then
+	echo ":::  -wild, --wildcard		Add whitecard entry (only blacklist)"
+fi
+	exit 0
+}
+
+EscapeRegexp() {
+    # This way we may safely insert an arbitrary
+    # string in our regular expressions
+    # Also remove leading "." if present
+    echo $* | sed 's/^\.*//' | sed "s/[]\.|$(){}?+*^]/\\\\&/g" | sed "s/\\//\\\\\//g"
 }
 
 HandleOther(){
-  #check validity of domain
-	validDomain=$(echo "$1" | perl -ne'print if /\b((?=[a-z0-9-]{1,63}\.)(xn--)?[a-z0-9]+(-[a-z0-9]+)*\.)+[a-z]{2,63}\b/')
+	# First, convert everything to lowercase
+	domain=$(sed -e "y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/" <<< "$1")
+
+	#check validity of domain
+	validDomain=$(echo "${domain}" | perl -lne 'print if /(?!.*[^a-z0-9-\.].*)^((?=[a-z0-9-]{1,63}\.)(xn--)?[a-z0-9-]+\.)*[a-z]{2,63}/')
 	if [ -z "${validDomain}" ]; then
 		echo "::: $1 is not a valid argument or domain name"
 	else
@@ -77,42 +91,93 @@ PoplistFile() {
 }
 
 AddDomain() {
-
 	list="$2"
+    domain=$(EscapeRegexp "$1")
 
-	bool=true
-	#Is the domain in the list we want to add it to?
-	grep -Ex -q "$1" ${list} > /dev/null 2>&1 || bool=false
+    if [[ "${list}" == "${whitelist}" || "${list}" == "${blacklist}" ]]; then
 
-	if [[ "${bool}" == false ]]; then
-	  #domain not found in the whitelist file, add it!
-	  if [[ "${verbose}" == true ]]; then
-		echo "::: Adding $1 to $list..."
-	  fi
-	  reload=true
-	  # Add it to the list we want to add it to
-	  echo "$1" >> ${list}
-	else
-        if [[ "${verbose}" == true ]]; then
-            echo "::: ${1} already exists in ${list}, no need to add!"
-        fi
+		bool=true
+		#Is the domain in the list we want to add it to?
+		grep -Ex -q "${domain}" "${list}" > /dev/null 2>&1 || bool=false
+
+		if [[ "${bool}" == false ]]; then
+		  #domain not found in the whitelist file, add it!
+		  if [[ "${verbose}" == true ]]; then
+			echo "::: Adding $1 to $list..."
+		  fi
+		  reload=true
+		  # Add it to the list we want to add it to
+		  echo "$1" >> "${list}"
+		else
+	        if [[ "${verbose}" == true ]]; then
+	            echo "::: ${1} already exists in ${list}, no need to add!"
+	        fi
+		fi
+
+	elif [[ "${list}" == "${wildcardlist}" ]]; then
+
+		source "${piholeDir}/setupVars.conf"
+		#Remove the /* from the end of the IPv4addr.
+		IPV4_ADDRESS=${IPV4_ADDRESS%/*}
+		IPV6_ADDRESS=${IPV6_ADDRESS}
+
+		bool=true
+		#Is the domain in the list?
+		grep -e "address=\/${domain}\/" "${wildcardlist}" > /dev/null 2>&1 || bool=false
+
+		if [[ "${bool}" == false ]]; then
+		  if [[ "${verbose}" == true ]]; then
+			echo "::: Adding $1 to wildcard blacklist..."
+		  fi
+		  reload=true
+		  echo "address=/$1/${IPV4_ADDRESS}" >> "${wildcardlist}"
+		  if [[ ${#IPV6_ADDRESS} > 0 ]] ; then
+		    echo "address=/$1/${IPV6_ADDRESS}" >> "${wildcardlist}"
+		  fi
+		else
+	        if [[ "${verbose}" == true ]]; then
+	            echo "::: ${1} already exists in wildcard blacklist, no need to add!"
+	        fi
+		fi
 	fi
 }
 
 RemoveDomain() {
     list="$2"
+    domain=$(EscapeRegexp "$1")
 
-    bool=true
-    #Is it in the other list? Logic follows that if its whitelisted it should not be blacklisted and vice versa
-    grep -Ex -q "$1" ${list} > /dev/null 2>&1 || bool=false
-    if [[ "${bool}" == true ]]; then
-        # Remove it from the other one
-        echo "::: Removing $1 from $list..."
-        echo "$1" | sed 's/\./\\./g' | xargs -I {} perl -i -ne'print unless /'{}'(?!.)/;' ${list}
-        reload=true
-    else
-        if [[ "${verbose}" == true ]]; then
-            echo "::: ${1} does not exist in ${list}, no need to remove!"
+    if [[ "${list}" == "${whitelist}" || "${list}" == "${blacklist}" ]]; then
+
+        bool=true
+        #Is it in the list? Logic follows that if its whitelisted it should not be blacklisted and vice versa
+        grep -Ex -q "${domain}" "${list}" > /dev/null 2>&1 || bool=false
+        if [[ "${bool}" == true ]]; then
+            # Remove it from the other one
+            echo "::: Removing $1 from $list..."
+            # /I flag: search case-insensitive
+            sed -i "/${domain}/Id" "${list}"
+            reload=true
+        else
+            if [[ "${verbose}" == true ]]; then
+                echo "::: ${1} does not exist in ${list}, no need to remove!"
+            fi
+        fi
+
+    elif [[ "${list}" == "${wildcardlist}" ]]; then
+
+        bool=true
+        #Is it in the list?
+        grep -e "address=\/${domain}\/" "${wildcardlist}" > /dev/null 2>&1 || bool=false
+        if [[ "${bool}" == true ]]; then
+            # Remove it from the other one
+            echo "::: Removing $1 from $list..."
+            # /I flag: search case-insensitive
+            sed -i "/address=\/${domain}/Id" "${list}"
+            reload=true
+        else
+            if [[ "${verbose}" == true ]]; then
+                echo "::: ${1} does not exist in ${list}, no need to remove!"
+            fi
         fi
     fi
 }
@@ -142,6 +207,7 @@ for var in "$@"; do
 	case "${var}" in
 	    "-w" | "whitelist"   ) listMain="${whitelist}"; listAlt="${blacklist}";;
 	    "-b" | "blacklist"   ) listMain="${blacklist}"; listAlt="${whitelist}";;
+		"-wild" | "wildcard" ) listMain="${wildcardlist}";;
 		"-nr"| "--noreload"  ) reload=false;;
 		"-d" | "--delmode"   ) addmode=false;;
 		"-f" | "--force"     ) force=true;;

advanced/Scripts/piholeDebug.sh

@@ -27,7 +27,7 @@ PIHOLELOG="/var/log/pihole.log"
 WHITELISTMATCHES="/tmp/whitelistmatches.list"
 
 IPV6_READY=false
-
+TIMEOUT=60
 # Header info and introduction
 cat << EOM
 ::: Beginning Pi-hole debug at $(date)!
@@ -122,6 +122,13 @@ version_check() {
 	&& log_echo -r "${light_ver}" || (log_echo "lighttpd not installed." && error_found=1)
 	local php_ver="$(php -v |& head -n1)" \
 	&& log_echo -r "${php_ver}" || (log_echo "PHP not installed." && error_found=1)
+
+	(local pi_hole_branch="$(cd /etc/.pihole/ && git rev-parse --abbrev-ref HEAD)" && log_echo -r "Pi-hole branch:  ${pi_hole_branch}") || log_echo "Unable to obtain Pi-hole branch"
+	(local pi_hole_rev="$(cd /etc/.pihole/ && git describe --long --dirty --tags)" && log_echo -r "Pi-hole rev:     ${pi_hole_rev}") || log_echo "Unable to obtain Pi-hole revision"
+
+	(local admin_branch="$(cd /var/www/html/admin && git rev-parse --abbrev-ref HEAD)" && log_echo -r "AdminLTE branch: ${admin_branch}") || log_echo "Unable to obtain AdminLTE branch"
+	(local admin_rev="$(cd /var/www/html/admin && git describe --long --dirty --tags)" && log_echo -r "AdminLTE rev:    ${admin_rev}") || log_echo "Unable to obtain AdminLTE revision"
+
 	return "${error_found}"
 }
 
@@ -316,6 +323,16 @@ debugLighttpd() {
   echo ":::"
 }
 
+countdown() {
+  tuvix=${TIMEOUT}
+  printf "::: Logging will automatically teminate in ${TIMEOUT} seconds\n"
+  while [ $tuvix -ge 1 ]
+  do
+    printf ":::\t${tuvix} seconds left. \r"
+    sleep 5
+    tuvix=$(( tuvix - 5 ))
+  done
+}
 ### END FUNCTIONS ###
 
 # Gather version of required packages / repositories
@@ -344,10 +361,21 @@ files_check "${ADLISTFILE}"
 
 header_write "Analyzing gravity.list"
 
-	gravity_length=$(wc -l "${GRAVITYFILE}") \
+	gravity_length=$(grep -c ^ "${GRAVITYFILE}") \
 	&& log_write "${GRAVITYFILE} is ${gravity_length} lines long." \
 	|| log_echo "Warning: No gravity.list file found!"
 
+header_write "Analyzing pihole.log"
+
+  pihole_length=$(grep -c ^ "${PIHOLELOG}") \
+  && log_write "${PIHOLELOG} is ${pihole_length} lines long." \
+  || log_echo "Warning: No pihole.log file found!"
+
+  pihole_size=$(du -h "${PIHOLELOG}" | awk '{ print $1 }') \
+  && log_write "${PIHOLELOG} is ${pihole_size}." \
+  || log_echo "Warning: No pihole.log file found!"
+
+
 # Continuously append the pihole.log file to the pihole_debug.log file
 dumpPiHoleLog() {
 	trap '{ echo -e "\n::: Finishing debug write from interrupt... Quitting!" ; exit 1; }' INT
@@ -356,10 +384,9 @@ dumpPiHoleLog() {
 	echo -e "::: Try loading a site that you are having trouble with now from a client web browser.. \n:::\t(Press CTRL+C to finish logging.)"
 	header_write "pihole.log"
 	if [ -e "${PIHOLELOG}" ]; then
-		while true; do
-			tail -f "${PIHOLELOG}" >> ${DEBUG_LOG}
-			log_write ""
-		done
+	# Dummy process to use for flagging down tail to terminate
+	  countdown &
+		tail -n0 -f --pid=$! "${PIHOLELOG}" >> ${DEBUG_LOG}
 	else
 		log_write "No pihole.log file found!"
 		printf ":::\tNo pihole.log file found!\n"
@@ -384,10 +411,13 @@ finalWork() {
 	# Check if tricorder.pi-hole.net is reachable and provide token.
 	if [ -n "${tricorder}" ]; then
 		echo "::: Your debug token is : ${tricorder}"
-		echo "::: Please contact the Pi-hole team with your token to being assistance."
+		echo "::: Please contact the Pi-hole team with your token for assistance."
 		echo "::: Thank you."
+	else
+		echo "::: There was an error uploading your debug log."
+		echo "::: Please try again or contact the Pi-hole team for assistance."
 	fi
-		echo "::: Debug log can be found at : /var/log/pihole_debug.log"
+		echo "::: A local copy of the Debug log can be found at : /var/log/pihole_debug.log"
 }
 
 trap finalWork EXIT

advanced/Scripts/piholeLogFlush.sh

@@ -11,5 +11,10 @@
 # (at your option) any later version.
 
 echo -n "::: Flushing /var/log/pihole.log ..."
-echo " " > /var/log/pihole.log
+# Test if logrotate is available on this system
+if command -v /usr/sbin/logrotate &> /dev/null; then
+  /usr/sbin/logrotate --force /etc/pihole/logrotate
+else
+  echo " " > /var/log/pihole.log
+fi
 echo "... done!"

advanced/Scripts/update.sh

@@ -3,7 +3,9 @@
 # (c) 2015, 2016 by Jacob Salmela
 # Network-wide ad blocking via your Raspberry Pi
 # http://pi-hole.net
-# Whitelists domains
+# Check Pi-hole core and admin pages versions and determine what
+# upgrade (if any) is required. Automatically updates and reinstalls
+# application if update is detected.
 #
 # Pi-hole is free software: you can redistribute it and/or modify
 # it under the terms of the GNU General Public License as published by
@@ -12,131 +14,206 @@
 
 # Variables
 
-webInterfaceGitUrl="https://github.com/pi-hole/AdminLTE.git"
-webInterfaceDir="/var/www/html/admin"
-piholeGitUrl="https://github.com/pi-hole/pi-hole.git"
-piholeFilesDir="/etc/.pihole"
-
-spinner() {
-	local pid=${1}
-	local delay=0.50
-	local spinstr='/-\|'
-	while [ "$(ps a | awk '{print $1}' | grep "${pid}")" ]; do
-		local temp=${spinstr#?}
-		printf " [%c]  " "${spinstr}"
-		local spinstr=${temp}${spinstr%"$temp"}
-		sleep ${delay}
-		printf "\b\b\b\b\b\b"
-	done
-	printf "    \b\b\b\b"
-}
-
-getGitFiles() {
-	# Setup git repos for directory and repository passed
-	# as arguments 1 and 2
-	echo ":::"
-	echo "::: Checking for existing repository..."
-	if is_repo "${1}"; then
-		update_repo "${1}"
-	else
-		make_repo "${1}" "${2}"
-	fi
-}
+readonly ADMIN_INTERFACE_GIT_URL="https://github.com/pi-hole/AdminLTE.git"
+readonly ADMIN_INTERFACE_DIR="/var/www/html/admin"
+readonly PI_HOLE_GIT_URL="https://github.com/pi-hole/pi-hole.git"
+readonly PI_HOLE_FILES_DIR="/etc/.pihole"
 
 is_repo() {
-	# Use git to check if directory is currently under VCS
-	echo -n ":::    Checking $1 is a repo..."
-	cd "${1}" &> /dev/null || return 1
-	git status &> /dev/null && echo " OK!"; return 0 || echo " not found!"; return 1
+  # Use git to check if directory is currently under VCS, return the value
+  local directory="${1}"
+  local curdir
+  local rc
+
+  curdir="${PWD}"
+  cd "${directory}" &> /dev/null || return 1
+  git status --short &> /dev/null
+  rc=$?
+  cd "${curdir}" &> /dev/null || return 1
+  return "${rc}"
+}
+
+prep_repo() {
+  # Prepare directory for local repository building
+  local directory="${1}"
+
+  rm -rf "${directory}" &> /dev/null
+  return
 }
 
 make_repo() {
-	# Remove the non-repod interface and clone the interface
-	echo -n ":::    Cloning $2 into $1..."
-	rm -rf "${1}"
-	git clone -q --depth 1 "${2}" "${1}" > /dev/null & spinner $!
-	echo " done!"
+  # Remove the non-repod interface and clone the interface
+  local remoteRepo="${2}"
+  local directory="${1}"
+
+  (prep_repo "${directory}" && git clone -q --depth 1 "${remoteRepo}" "${directory}")
+     return
 }
 
 update_repo() {
-# Pull the latest commits
-	echo -n ":::     Updating repo in $1..."
-	cd "${1}" || exit 1
-	git stash -q > /dev/null & spinner $!
-	git pull -q > /dev/null & spinner $!
-	echo " done!"
+  local directory="${1}"
+  local curdir
+
+  curdir="${PWD}"
+  cd "${directory}" &> /dev/null || return 1
+  # Pull the latest commits
+  # Stash all files not tracked for later retrieval
+  git stash --all --quiet
+  # Force a clean working directory for cloning
+  git clean --force -d
+  # Fetch latest changes and apply
+  git pull --quiet
+  cd "${curdir}" &> /dev/null || return 1
 }
 
-if [ ! -d "/etc/.pihole" ]; then #This is unlikely
-	echo "::: Critical Error: Pi-Hole repo missing from system!"
-	echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
-	exit 1;
-fi
-if [ ! -d "/var/www/html/admin" ]; then #This is unlikely
-	echo "::: Critical Error: Pi-Hole repo missing from system!"
-	echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
-	exit 1;
-fi
+getGitFiles() {
+  # Setup git repos for directory and repository passed
+  # as arguments 1 and 2
+  local directory="${1}"
+  local remoteRepo="${2}"
+  echo ":::"
+  echo "::: Checking for existing repository..."
+  if is_repo "${directory}"; then
+    echo -n ":::     Updating repository in ${directory}..."
+    update_repo "${directory}" || (echo "*** Error: Could not update local repository. Contact support."; exit 1)
+    echo " done!"
+  else
+    echo -n ":::    Cloning ${remoteRepo} into ${directory}..."
+    make_repo "${directory}" "${remoteRepo}" || (echo "Unable to clone repository, please contact support"; exit 1)
+    echo " done!"
+  fi
+}
 
-echo "::: Checking for updates..."
-piholeVersion=$(pihole -v -p -c)
-piholeVersionLatest=$(pihole -v -p -l)
+GitCheckUpdateAvail() {
+  local directory="${1}"
+  curdir=$PWD;
+  cd "${directory}"
 
-webVersion=$(pihole -v -a -c)
-webVersionLatest=$(pihole -v -a -l)
+  # Fetch latest changes in this repo
+  git fetch --quiet origin
 
-echo ":::"
-echo "::: Pi-hole version is $piholeVersion (Latest version is $piholeVersionLatest)"
-echo "::: Web Admin version is $webVersion (Latest version is $webVersionLatest)"
-echo ":::"
+  # @ alone is a shortcut for HEAD. Older versions of git
+  # need @{0}
+  LOCAL="$(git rev-parse @{0})"
 
-# Logic
-# If latest versions are blank - we've probably hit Github rate limit (stop running `pihole -up so often!):
-#            Update anyway
-# If Core up to date AND web up to date:
-#            Do nothing
-# If Core up to date AND web NOT up to date:
-#            Pull web repo
-# If Core NOT up to date AND web up to date:
-#            pull pihole repo, run install --unattended -- reconfigure
-# if Core NOT up to date AND web NOT up to date:
-#            pull pihole repo run install --unattended
+  # The suffix @{upstream} to a branchname
+  # (short form <branchname>@{u}) refers
+  # to the branch that the branch specified
+  # by branchname is set to build on top of#
+  # (configured with branch.<name>.remote and
+  # branch.<name>.merge). A missing branchname
+  # defaults to the current one.
+  REMOTE="$(git rev-parse @{upstream})"
 
+  if [[ ${#LOCAL} == 0 ]]; then
+    echo "::: Error: Local revision could not be obtained, ask Pi-hole support."
+    echo "::: Additional debugging output:"
+    git status
+    exit
+  fi
+  if [[ ${#REMOTE} == 0 ]]; then
+    echo "::: Error: Remote revision could not be obtained, ask Pi-hole support."
+    echo "::: Additional debugging output:"
+    git status
+    exit
+  fi
+  
+  # Change back to original directory
+  cd "${curdir}"
 
+  if [[ "${LOCAL}" != "${REMOTE}" ]]; then
+    # Local branch is behind remote branch -> Update
+    return 0
+  else
+    # Local branch is up-to-date or in a situation
+    # where this updater cannot be used (like on a
+    # branch that exists only locally)
+    return 1
+  fi
+}
 
-if [[ ${piholeVersion} == ${piholeVersionLatest} && ${webVersion} == ${webVersionLatest} ]]; then
-	echo "::: Everything is up to date!"
-	echo ""
-	exit 0
+main() {
+  local pihole_version_current
+  local web_version_current
 
-elif [[ ${piholeVersion} == ${piholeVersionLatest} && ${webVersion} != ${webVersionLatest} ]]; then
-	echo "::: Pi-hole Web Admin files out of date"
-	getGitFiles ${webInterfaceDir} ${webInterfaceGitUrl}
-	echo ":::"
-	webVersion=$(pihole -v -a -c)
-	echo "::: Web Admin version is now at ${webVersion}"
-	echo "::: If you had made any changes in '/var/www/html/admin', they have been stashed using 'git stash'"
-	echo ""
-elif [[ ${piholeVersion} != ${piholeVersionLatest} && ${webVersion} == ${webVersionLatest} ]]; then
-	echo "::: Pi-hole core files out of date"
-	getGitFiles ${piholeFilesDir} ${piholeGitUrl}
-	/etc/.pihole/automated\ install/basic-install.sh --reconfigure --unattended
-	echo ":::"
-	piholeVersion=$(pihole -v -p -c)
-	echo "::: Pi-hole version is now at ${piholeVersion}"
-	echo "::: If you had made any changes in '/etc/.pihole', they have been stashed using 'git stash'"
-	echo ""
-elif [[ ${piholeVersion} != ${piholeVersionLatest} && ${webVersion} != ${webVersionLatest} ]]; then
-	echo "::: Updating Everything"
-	getGitFiles ${piholeFilesDir} ${piholeGitUrl}
-	/etc/.pihole/automated\ install/basic-install.sh --unattended
-	webVersion=$(pihole -v -a -c)
-	piholeVersion=$(pihole -v -p -c)
-	echo ":::"
-	echo "::: Pi-hole version is now at ${piholeVersion}"
-	echo "::: If you had made any changes in '/etc/.pihole', they have been stashed using 'git stash'"
-	echo ":::"
-	echo "::: Pi-hole version is now at ${piholeVersion}"
-	echo "::: If you had made any changes in '/etc/.pihole', they have been stashed using 'git stash'"
-	echo ""
-fi
+  #This is unlikely
+  if ! is_repo "${PI_HOLE_FILES_DIR}" || ! is_repo "${ADMIN_INTERFACE_DIR}" ; then
+    echo "::: Critical Error: One or more Pi-Hole repos are missing from system!"
+    echo "::: Please re-run install script from https://github.com/pi-hole/pi-hole"
+    exit 1;
+  fi
+
+  echo "::: Checking for updates..."
+
+  if GitCheckUpdateAvail "${PI_HOLE_FILES_DIR}" ; then
+    core_update=true
+    echo "::: Pi-hole Core:   update available"
+  else
+    core_update=false
+    echo "::: Pi-hole Core:   up to date"
+  fi
+
+  if GitCheckUpdateAvail "${ADMIN_INTERFACE_DIR}" ; then
+    web_update=true
+    echo "::: Web Interface:  update available"
+  else
+    web_update=false
+    echo "::: Web Interface:  up to date"
+  fi
+
+  # Logic
+  # If Core up to date AND web up to date:
+  #            Do nothing
+  # If Core up to date AND web NOT up to date:
+  #            Pull web repo
+  # If Core NOT up to date AND web up to date:
+  #            pull pihole repo, run install --unattended -- reconfigure
+  # if Core NOT up to date AND web NOT up to date:
+  #            pull pihole repo run install --unattended
+
+  if ! ${core_update} && ! ${web_update} ; then
+    echo ":::"
+    echo "::: Everything is up to date!"
+    exit 0
+
+  elif ! ${core_update} && ${web_update} ; then
+    echo ":::"
+    echo "::: Pi-hole Web Admin files out of date"
+    getGitFiles "${ADMIN_INTERFACE_DIR}" "${ADMIN_INTERFACE_GIT_URL}"
+
+  elif ${core_update} && ! ${web_update} ; then
+    echo ":::"
+    echo "::: Pi-hole core files out of date"
+    getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
+    /etc/.pihole/automated\ install/basic-install.sh --reconfigure --unattended || echo "Unable to complete update, contact Pi-hole" && exit 1
+
+  elif ${core_update} && ${web_update} ; then
+    echo ":::"
+    echo "::: Updating Everything"
+    getGitFiles "${PI_HOLE_FILES_DIR}" "${PI_HOLE_GIT_URL}"
+    /etc/.pihole/automated\ install/basic-install.sh --unattended || echo "Unable to complete update, contact Pi-hole" && exit 1
+  else
+    echo "*** Update script has malfunctioned, fallthrough reached. Please contact support"
+    exit 1
+  fi
+
+  if [[ "${web_update}" == true ]]; then
+    web_version_current="$(/usr/local/bin/pihole version --admin --current)"
+    echo ":::"
+    echo "::: Web Admin version is now at ${web_version_current}"
+    echo "::: If you had made any changes in '/var/www/html/admin/', they have been stashed using 'git stash'"
+  fi
+
+  if [[ "${core_update}" == true ]]; then
+    pihole_version_current="$(/usr/local/bin/pihole version --pihole --current)"
+    echo ":::"
+    echo "::: Pi-hole version is now at ${pihole_version_current}"
+    echo "::: If you had made any changes in '/etc/.pihole/', they have been stashed using 'git stash'"
+  fi
+
+  echo ""
+  exit 0
+
+}
+
+main

advanced/Scripts/version.sh

@@ -14,6 +14,8 @@
 latest=false
 current=false
 
+DEFAULT="-1"
+
 normalOutput() {
 	piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
 	webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
@@ -21,8 +23,8 @@ normalOutput() {
 	piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
 	webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
 
-	echo "::: Pi-hole version is ${piholeVersion} (Latest version is ${piholeVersionLatest})"
-	echo "::: Web-Admin version is ${webVersion} (Latest version is ${webVersionLatest})"
+	echo "::: Pi-hole version is ${piholeVersion} (Latest version is ${piholeVersionLatest:-${DEFAULT}})"
+	echo "::: Web-Admin version is ${webVersion} (Latest version is ${webVersionLatest:-${DEFAULT}})"
 }
 
 webOutput() {
@@ -36,14 +38,14 @@ webOutput() {
 
 	if [[ "${latest}" == true && "${current}" == false ]]; then
 		webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo ${webVersionLatest}
+		echo "${webVersionLatest:--1}"
 	elif [[ "${latest}" == false && "${current}" == true ]]; then
 		webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
-		echo ${webVersion}
+		echo "${webVersion}"
 	else
 		webVersion=$(cd /var/www/html/admin/ && git describe --tags --abbrev=0)
 		webVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/AdminLTE/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "::: Web-Admin version is $webVersion (Latest version is $webVersionLatest)"
+		echo "::: Web-Admin version is ${webVersion} (Latest version is ${webVersionLatest:-${DEFAULT}})"
 	fi
 }
 
@@ -58,14 +60,14 @@ coreOutput() {
 
 	if [[ "${latest}" == true && "${current}" == false ]]; then
 		piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo ${piholeVersionLatest}
+		echo "${piholeVersionLatest:--1}"
 	elif [[ "${latest}" == false && "${current}" == true ]]; then
 		piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
-		echo ${piholeVersion}
+		echo "${piholeVersion}"
 	else
 		piholeVersion=$(cd /etc/.pihole/ && git describe --tags --abbrev=0)
 		piholeVersionLatest=$(curl -s https://api.github.com/repos/pi-hole/pi-hole/releases/latest | grep -Po '"tag_name":.*?[^\\]",' |  perl -pe 's/"tag_name": "//; s/^"//; s/",$//')
-		echo "::: Pi-hole version is $piholeVersion (Latest version is $piholeVersionLatest)"
+		echo "::: Pi-hole version is ${piholeVersion} (Latest version is ${piholeVersionLatest:-${DEFAULT}})"
 	fi
 }
 
@@ -84,7 +86,7 @@ helpFunc() {
 :::  -h, --help           Show this help dialog
 :::
 EOM
-	exit 1
+	exit 0
 }
 
 if [[ $# = 0 ]]; then

advanced/Scripts/webpage.sh

@@ -0,0 +1,346 @@
+#!/usr/bin/env bash
+# Pi-hole: A black hole for Internet advertisements
+# Network-wide ad blocking via your Raspberry Pi
+# http://pi-hole.net
+# Web interface settings
+#
+# Pi-hole is free software: you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation, either version 2 of the License, or
+# (at your option) any later version.
+
+readonly setupVars="/etc/pihole/setupVars.conf"
+readonly dnsmasqconfig="/etc/dnsmasq.d/01-pihole.conf"
+readonly dhcpconfig="/etc/dnsmasq.d/02-pihole-dhcp.conf"
+
+helpFunc() {
+	cat << EOM
+::: Set admin options for the web interface of pihole
+:::
+::: Usage: pihole -a [options]
+:::
+::: Options:
+:::  -p, password		Set web interface password, an empty input will remove any previously set password
+:::  -c, celsius		Set Celsius temperature unit
+:::  -f, fahrenheit		Set Fahrenheit temperature unit
+:::  -k, kelvin			Set Kelvin temperature unit
+:::  -h, --help			Show this help dialog
+EOM
+	exit 0
+}
+
+add_setting() {
+	echo "${1}=${2}" >> "${setupVars}"
+}
+
+delete_setting() {
+	sed -i "/${1}/d" "${setupVars}"
+}
+
+change_setting() {
+	delete_setting "${1}"
+	add_setting "${1}" "${2}"
+}
+
+add_dnsmasq_setting() {
+	if [[ "${2}" != "" ]]; then
+		echo "${1}=${2}" >> "${dnsmasqconfig}"
+	else
+		echo "${1}" >> "${dnsmasqconfig}"
+	fi
+}
+
+delete_dnsmasq_setting() {
+	sed -i "/${1}/d" "${dnsmasqconfig}"
+}
+
+SetTemperatureUnit(){
+
+	change_setting "TEMPERATUREUNIT" "${unit}"
+
+}
+
+SetWebPassword(){
+
+	if [ "${SUDO_USER}" == "www-data" ]; then
+		echo "Security measure: user www-data is not allowed to change webUI password!"
+		echo "Exiting"
+		exit 1
+	fi
+
+	if [ "${SUDO_USER}" == "lighttpd" ]; then
+		echo "Security measure: user lighttpd is not allowed to change webUI password!"
+		echo "Exiting"
+		exit 1
+	fi
+
+	# Set password only if there is one to be set
+	if (( ${#args[2]} > 0 )) ; then
+		# Compute password hash twice to avoid rainbow table vulnerability
+		hash=$(echo -n ${args[2]} | sha256sum | sed 's/\s.*$//')
+		hash=$(echo -n ${hash} | sha256sum | sed 's/\s.*$//')
+		# Save hash to file
+		change_setting "WEBPASSWORD" "${hash}"
+		echo "New password set"
+	else
+		change_setting "WEBPASSWORD" ""
+		echo "Password removed"
+	fi
+
+}
+
+ProcessDNSSettings() {
+	source "${setupVars}"
+
+	delete_dnsmasq_setting "server"
+
+	COUNTER=1
+	while [[ 1 ]]; do
+		var=PIHOLE_DNS_${COUNTER}
+		if [ -z "${!var}" ]; then
+			break;
+		fi
+		add_dnsmasq_setting "server" "${!var}"
+		let COUNTER=COUNTER+1
+	done
+
+	delete_dnsmasq_setting "domain-needed"
+
+	if [[ "${DNS_FQDN_REQUIRED}" == true ]]; then
+		add_dnsmasq_setting "domain-needed"
+	fi
+
+	delete_dnsmasq_setting "bogus-priv"
+
+	if [[ "${DNS_BOGUS_PRIV}" == true ]]; then
+		add_dnsmasq_setting "bogus-priv"
+	fi
+
+	delete_dnsmasq_setting "dnssec"
+	delete_dnsmasq_setting "trust-anchor="
+
+	if [[ "${DNSSEC}" == true ]]; then
+		echo "dnssec
+trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5
+" >> "${dnsmasqconfig}"
+	fi
+
+}
+
+SetDNSServers(){
+
+	# Save setting to file
+	delete_setting "PIHOLE_DNS"
+	IFS=',' read -r -a array <<< "${args[2]}"
+	for index in "${!array[@]}"
+	do
+		add_setting "PIHOLE_DNS_$((index+1))" "${array[index]}"
+	done
+
+	if [[ "${args[3]}" == "domain-needed" ]]; then
+		change_setting "DNS_FQDN_REQUIRED" "true"
+	else
+		change_setting "DNS_FQDN_REQUIRED" "false"
+	fi
+
+	if [[ "${args[4]}" == "bogus-priv" ]]; then
+		change_setting "DNS_BOGUS_PRIV" "true"
+	else
+		change_setting "DNS_BOGUS_PRIV" "false"
+	fi
+
+	if [[ "${args[5]}" == "dnssec" ]]; then
+		change_setting "DNSSEC" "true"
+	else
+		change_setting "DNSSEC" "false"
+	fi
+
+	ProcessDNSSettings
+
+	# Restart dnsmasq to load new configuration
+	RestartDNS
+
+}
+
+SetExcludeDomains(){
+
+	change_setting "API_EXCLUDE_DOMAINS" "${args[2]}"
+
+}
+
+SetExcludeClients(){
+
+	change_setting "API_EXCLUDE_CLIENTS" "${args[2]}"
+
+}
+
+Reboot(){
+
+	nohup bash -c "sleep 5; reboot" &> /dev/null </dev/null &
+
+}
+
+RestartDNS(){
+
+	if [ -x "$(command -v systemctl)" ]; then
+		systemctl restart dnsmasq &> /dev/null
+	else
+		service dnsmasq restart &> /dev/null
+	fi
+
+}
+
+SetQueryLogOptions(){
+
+	change_setting "API_QUERY_LOG_SHOW" "${args[2]}"
+
+}
+
+ProcessDHCPSettings() {
+
+	source "${setupVars}"
+
+	if [[ "${DHCP_ACTIVE}" == "true" ]]; then
+
+	interface=$(grep 'PIHOLE_INTERFACE=' /etc/pihole/setupVars.conf | sed "s/.*=//")
+
+	# Use eth0 as fallback interface
+	if [ -z ${interface} ]; then
+		interface="eth0"
+	fi
+
+	if [[ "${PIHOLE_DOMAIN}" == "" ]]; then
+		PIHOLE_DOMAIN="local"
+		change_setting "PIHOLE_DOMAIN" "${PIHOLE_DOMAIN}"
+	fi
+
+	if [[ "${DHCP_LEASETIME}" == "0" ]]; then
+		leasetime="infinite"
+	elif [[ "${DHCP_LEASETIME}" == "" ]]; then
+		leasetime="24h"
+		change_setting "DHCP_LEASETIME" "${leasetime}"
+	else
+		leasetime="${DHCP_LEASETIME}h"
+	fi
+
+	# Write settings to file
+	echo "###############################################################################
+#  DHCP SERVER CONFIG FILE AUTOMATICALLY POPULATED BY PI-HOLE WEB INTERFACE.  #
+#            ANY CHANGES MADE TO THIS FILE WILL BE LOST ON CHANGE             #
+###############################################################################
+dhcp-authoritative
+dhcp-range=${DHCP_START},${DHCP_END},${leasetime}
+dhcp-option=option:router,${DHCP_ROUTER}
+dhcp-leasefile=/etc/pihole/dhcp.leases
+#quiet-dhcp
+" > "${dhcpconfig}"
+
+if [[ "${PIHOLE_DOMAIN}" != "none" ]]; then
+	echo "domain=${PIHOLE_DOMAIN}" >> "${dhcpconfig}"
+fi
+
+	if [[ "${DHCP_IPv6}" == "true" ]]; then
+echo "#quiet-dhcp6
+#enable-ra
+dhcp-option=option6:dns-server,[::]
+dhcp-range=::100,::1ff,constructor:${interface},ra-names,slaac,${leasetime}
+ra-param=*,0,0
+" >> "${dhcpconfig}"
+	fi
+
+	else
+		rm "${dhcpconfig}" &> /dev/null
+	fi
+}
+
+EnableDHCP(){
+
+	change_setting "DHCP_ACTIVE" "true"
+	change_setting "DHCP_START" "${args[2]}"
+	change_setting "DHCP_END" "${args[3]}"
+	change_setting "DHCP_ROUTER" "${args[4]}"
+	change_setting "DHCP_LEASETIME" "${args[5]}"
+	change_setting "PIHOLE_DOMAIN" "${args[6]}"
+	change_setting "DHCP_IPv6" "${args[7]}"
+
+	# Remove possible old setting from file
+	delete_dnsmasq_setting "dhcp-"
+	delete_dnsmasq_setting "quiet-dhcp"
+
+	ProcessDHCPSettings
+
+	RestartDNS
+}
+
+DisableDHCP(){
+
+	change_setting "DHCP_ACTIVE" "false"
+
+	# Remove possible old setting from file
+	delete_dnsmasq_setting "dhcp-"
+	delete_dnsmasq_setting "quiet-dhcp"
+
+	ProcessDHCPSettings
+
+	RestartDNS
+}
+
+SetWebUILayout(){
+
+	change_setting "WEBUIBOXEDLAYOUT" "${args[2]}"
+
+}
+
+SetPrivacyMode(){
+
+	if [[ "${args[2]}" == "true" ]] ; then
+		change_setting "API_PRIVACY_MODE" "true"
+	else
+		change_setting "API_PRIVACY_MODE" "false"
+	fi
+
+}
+
+ResolutionSettings() {
+
+	typ="${args[2]}"
+	state="${args[3]}"
+
+	if [[ "${typ}" == "forward" ]]; then
+		change_setting "API_GET_UPSTREAM_DNS_HOSTNAME" "${state}"
+	elif [[ "${typ}" == "clients" ]]; then
+		change_setting "API_GET_CLIENT_HOSTNAME" "${state}"
+	fi
+}
+
+main() {
+
+	args=("$@")
+
+	case "${args[1]}" in
+		"-p" | "password"   ) SetWebPassword;;
+		"-c" | "celsius"    ) unit="C"; SetTemperatureUnit;;
+		"-f" | "fahrenheit" ) unit="F"; SetTemperatureUnit;;
+		"-k" | "kelvin"     ) unit="K"; SetTemperatureUnit;;
+		"setdns"            ) SetDNSServers;;
+		"setexcludedomains" ) SetExcludeDomains;;
+		"setexcludeclients" ) SetExcludeClients;;
+		"reboot"            ) Reboot;;
+		"restartdns"        ) RestartDNS;;
+		"setquerylog"       ) SetQueryLogOptions;;
+		"enabledhcp"        ) EnableDHCP;;
+		"disabledhcp"       ) DisableDHCP;;
+		"layout"            ) SetWebUILayout;;
+		"-h" | "--help"     ) helpFunc;;
+		"privacymode"       ) SetPrivacyMode;;
+		"resolve"           ) ResolutionSettings;;
+		*                   ) helpFunc;;
+	esac
+
+	shift
+
+	if [[ $# = 0 ]]; then
+		helpFunc
+	fi
+
+}

advanced/bash-completion/pihole

@@ -3,7 +3,7 @@ _pihole() {
 	COMPREPLY=()
 	cur="${COMP_WORDS[COMP_CWORD]}"
 	prev="${COMP_WORDS[COMP_CWORD-1]}"
-	opts="blacklist chronometer debug flush help query reconfigure setupLCD uninstall updateGravity updatePihole version whitelist"
+	opts="admin blacklist chronometer debug disable enable flush help logging query reconfigure restartdns setupLCD status tail uninstall updateGravity updatePihole version whitelist"
 
 	COMPREPLY=( $(compgen -W "${opts}" -- ${cur}) )
 	return 0

advanced/blockingpage.css

@@ -0,0 +1,136 @@
+/* CSS Reset */
+html, body, div, span, applet, object, iframe, h1, h2, h3, h4, h5, h6, p, blockquote, pre, a, abbr, acronym, address, big, cite, code, del, dfn, em, img, ins, kbd, q, s, samp, small, strike, strong, sub, sup, tt, var, b, u, i, center, dl, dt, dd, ol, ul, li, fieldset, form, label, legend, table, caption, tbody, tfoot, thead, tr, th, td, article, aside, canvas, details, embed, figure, figcaption, footer, header, hgroup, menu, nav, output, ruby, section, summary, time, mark, audio, video { margin: 0; padding: 0; border: 0; font-size: 100%; font: inherit; vertical-align: baseline; }
+article, aside, details, figcaption, figure, footer, header, hgroup, menu, nav, section { display: block; }
+body { line-height: 1; }
+ol, ul { list-style: none; }
+blockquote, q { quotes: none; }
+blockquote:before, blockquote:after, q:before, q:after { content: ''; content: none; }
+table { border-collapse: collapse; border-spacing: 0; }
+html { height: 100%; overflow-x: hidden; }
+
+/* General Style */
+a	{ color: rgba(0,60,120,0.95); text-decoration: none; } /* 1E3C5A */
+a:hover { color: rgba(210,120,0,0.95); transition-duration: .2s; } /* 255, 128, 0 */
+divs a		{ border-bottom: 1px dashed rgba(30,60,90,0.3); }
+b	{ font-weight: bold; }
+i	{ font-style: italic; }
+
+footer, pre, td { font-family: monospace; padding-left: 15px; }
+/*body, header { background: #E1E1E1; }*/
+
+body {
+	background-image: -webkit-linear-gradient(top, rgba(240,240,240,0.95), rgba(190,190,190,0.95));
+	background-image: linear-gradient(to bottom, rgba(240,240,240,0.95), rgba(190,190,190,0.95));
+	background-attachment: fixed;
+	color: rgba(64,64,64,0.95);
+	font: 14px, sans-serif;
+	line-height: 1em;
+}
+
+header {
+	min-width: 320px;
+	width: 100%;
+	text-shadow: 0 1px rgba(255,255,255,0.6);
+	display: table;
+	table-layout: fixed;
+	border: 1px solid rgba(0,0,0,0.25);
+	border-top-color: rgba(255,255,255,0.85);
+	border-style: solid none;
+	background-image: -webkit-linear-gradient(top, rgba(240,240,240,0.95), rgba(220,220,220,0.95));
+	background-image: linear-gradient(to bottom, rgba(240,240,240,0.95), rgba(220,220,220,0.95));
+	box-shadow:  0 0 1px 1px rgba(0,0,0,0.04);
+}
+
+header h1, header div {
+	display: table-cell;
+	color: inherit;
+	font-weight: bold;
+	vertical-align: middle;
+	white-space: nowrap;
+	overflow: hidden;
+	box-sizing: border-box;
+}
+
+header h1 {
+	font-size: 22px;
+	font-weight: bold;
+	width: 100%;
+	padding: 8px 0;
+	text-indent: 32px;
+	background: url("http://pi.hole/admin/img/logo.svg") left no-repeat;
+	background-size: 30px 22px;
+}
+
+header h1 a, h1 a:hover { color: inherit; }
+header .alt { width: 85px; font-size: 0.8em; padding-right: 4px; text-align: right; line-height: 1.25em; }
+.active { color: green; }
+.inactive { color: red; }
+
+main {
+	display: block;
+	width: 80%;
+	padding: 10px;
+	font-size: 1em;
+	background-color: rgba(255,255,255,0.85);
+	margin: 8px auto;
+	box-sizing: border-box;
+	border: 1px solid rgba(0,0,0,0.25);
+	box-shadow:  4px 4px rgba(0,0,0,0.1);
+	line-height: 1.2em;
+	border-radius: 8px;
+}
+
+h2 { /* Rgba is shared with .transparent th */
+	font: 1.15em sans-serif;
+	background-color: rgba(255,0,0,0.4);
+	text-shadow: none;
+	line-height: 1.1em;
+	padding-bottom: 1px;
+	margin-top: 8px;
+	margin-bottom: 4px;
+	background: -webkit-linear-gradient(left, rgba(0,0,0,0.25), transparent 80%) no-repeat;
+	background: linear-gradient(to right, rgba(0,0,0,0.25), transparent 80%) no-repeat;
+	background-size: 100% 1px;
+	background-position: 0 17px;
+}
+
+h2:first-child { margin-top: 0; }
+h2 ~ *:not(h2) { margin-left: 4px; }
+li { padding: 2px 0; }
+li::before { content: "\00BB\00a0"; }
+li a { position: relative; top: 1px; } /* Center bullet-point arrows */
+
+/* Button Style */
+.buttons a, button, input, .transparent th a { /* Swapped rgba is shared with input[type='url'] */
+	display: inline-block;
+	color: rgba(32,32,32,0.9);
+	font-weight: bold;
+	text-align: center;
+	cursor: pointer;
+	text-shadow: 0 1px rgba(255,255,255,0.2);
+	line-height: 0.86em;
+	font-size: 1em;
+	padding: 4px 8px;
+	background: #FAFAFA;
+	background-image: -webkit-linear-gradient(top, rgba(255,255,255,0.05), rgba(0,0,0,0.05));
+	background-image: linear-gradient(to bottom, rgba(255,255,255,0.05), rgba(0,0,0,0.05));
+	border: 1px solid rgba(0,0,0,0.25);
+	border-radius: 4px;
+	box-shadow: 0 1px 0 rgba(0,0,0,0.04);
+}
+
+.buttons { white-space: nowrap; width: 100%; display: table; }
+.buttons33 { white-space: nowrap; width: 33.333%; display: table; text-align: center; margin-left: 33.333% }
+.mini a { width: 50%; }
+a.safe { background-color: rgba(0,220,0,0.5); }
+button.safe { background-color: rgba(0,220,0,0.5); }
+a.warn { background-color: rgba(220,0,0,0.5); }
+
+.blocked a, .mini a { display: table-cell; }
+.blocked a.safe50 { width: 50%; background-color: rgba(0,220,0,0.5); }
+.blocked a.safe33 { width: 33.333%; background-color: rgba(0,220,0,0.5); }
+
+/* Types of text */
+.msg { white-space: pre; overflow: auto; -webkit-overflow-scrolling: touch; display: block; line-height: 1.2em; font-weight: bold; font-size: 1.15em; margin: 4px 8px 8px 8px; white-space: pre-line; }
+
+footer { font-size: 0.8em; text-align: center; width: 87%; margin: 4px auto; }

advanced/dphys-swapfile

@@ -1,12 +0,0 @@
-# Pi-hole: A black hole for Internet advertisements
-# (c) 2015, 2016 by Jacob Salmela
-# Network-wide ad blocking via your Raspberry Pi
-# http://pi-hole.net
-# Swap file config
-#
-# Pi-hole is free software: you can redistribute it and/or modify
-# it under the terms of the GNU General Public License as published by
-# the Free Software Foundation, either version 2 of the License, or
-# (at your option) any later version.
-
-CONF_SWAPSIZE=500

advanced/index.html

@@ -1,7 +0,0 @@
-<html>
-<head>
-<script>window.close();</script>
-</head>
-<body>
-</body>
-</html>

advanced/index.php

@@ -0,0 +1,183 @@
+<?php
+/* Detailed Pi-Hole Block Page: Show "Website Blocked" if user browses to site, but not to image/file requests based on the work of WaLLy3K for DietPi & Pi-Hole */
+
+$uri = escapeshellcmd($_SERVER['REQUEST_URI']);
+$serverName = escapeshellcmd($_SERVER['SERVER_NAME']);
+
+// Retrieve server URI extension (EG: jpg, exe, php)
+$uriExt = pathinfo($uri, PATHINFO_EXTENSION);
+
+// Define which URL extensions get rendered as "Website Blocked"
+$webExt = array('asp', 'htm', 'html', 'php', 'rss', 'xml');
+
+if(in_array($uriExt, $webExt) || empty($uriExt))
+{
+	// Requested resource has an extension listed in $webExt
+	// or no extension (index access to some folder incl. the root dir)
+	$showPage = true;
+}
+else
+{
+	// Something else
+	$showPage = false;
+}
+
+// Handle incoming URI types
+if (!$showPage)
+{
+?>
+<html>
+<head>
+<script>window.close();</script></head>
+<body>
+<img src="data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7">
+</body>
+</html>
+<?php
+	die();
+}
+
+// Get Pi-Hole version
+$piHoleVersion = exec('cd /etc/.pihole/ && git describe --tags --abbrev=0');
+
+// Don't show the URI if it is the root directory
+if($uri == "/")
+{
+	$uri = "";
+}
+
+?>
+<!DOCTYPE html>
+<head>
+	<meta charset='UTF-8'/>
+	<title>Website Blocked</title>
+	<link rel='stylesheet' href='http://pi.hole/pihole/blockingpage.css'/>
+	<link rel='shortcut icon' href='http://pi.hole/admin/img/favicon.png' type='image/png'/>
+	<meta name='viewport' content='width=device-width,initial-scale=1.0,maximum-scale=1.0, user-scalable=no'/>
+	<meta name='robots' content='noindex,nofollow'/>
+</head>
+<body id="body">
+<header>
+	<h1><a href='/'>Website Blocked</a></h1>
+</header>
+<main>
+	<div>Access to the following site has been blocked:<br/>
+	<span class='pre msg'><?php echo $serverName.$uri; ?></span></div>
+	<div>If you have an ongoing use for this website, please ask the owner of the Pi-hole in your network to have it whitelisted.</div>
+	<input id="domain" type="hidden" value="<?php echo $serverName; ?>">
+	<input id="quiet" type="hidden" value="yes">
+	<button id="btnSearch" class="buttons blocked" type="button" style="visibility: hidden;"></button>
+	This page is blocked because it is explicitly contained within the following block list(s):
+	<pre id="output" style="width: 100%; height: 100%;" hidden="true"></pre><br/>
+	<div class='buttons blocked'>
+		<a class='safe33' href='javascript:history.back()'>Go back</a>
+		<a class='safe33' id="whitelisting">Whitelist this page</a>
+		<a class='safe33' href='javascript:window.close()'>Close window</a>
+	</div>
+		<div style="width: 98%; text-align: center; padding: 10px;" hidden="true" id="whitelistingform">
+			<p>Note that whitelisting domains which are blocked using the wildcard method won't work.</p>
+			<p>Password required!</p><br/>
+		<form>
+			<input name="list" type="hidden" value="white"><br/>
+			Domain:<br/>
+			<input name="domain" value="<?php echo $serverName ?>" disabled><br/><br/>
+			Password:<br/>
+			<input type="password" id="pw" name="pw"><br/><br/>
+			<button class="buttons33 safe" id="btnAdd" type="button">Whitelist</button>
+		</form><br/>
+		<pre id="whitelistingoutput" style="width: 100%; height: 100%; padding: 5px;" hidden="true"></pre><br/>
+		</div>
+</main>
+<footer>Generated <?php echo date('D g:i A, M d'); ?> by Pi-hole <?php echo $piHoleVersion; ?></footer>
+<script src="http://pi.hole/admin/scripts/vendor/jquery.min.js"></script>
+<script>
+// Create event for when the output is appended to
+(function($) {
+    var origAppend = $.fn.append;
+
+    $.fn.append = function () {
+        return origAppend.apply(this, arguments).trigger("append");
+    };
+})(jQuery);
+</script>
+<script src="http://pi.hole/admin/scripts/pi-hole/js/queryads.js"></script>
+<script>
+function inIframe () {
+    try {
+        return window.self !== window.top;
+    } catch (e) {
+        return true;
+    }
+}
+
+// Try to detect if page is loaded within iframe
+if(inIframe())
+{
+    // Within iframe
+    // hide content of page
+    $('#body').hide();
+    // remove background
+    document.body.style.backgroundImage = "none";
+}
+else
+{
+    // Query adlists
+    $( "#btnSearch" ).click();
+}
+
+$( "#whitelisting" ).on( "click", function(){ $( "#whitelistingform" ).removeAttr( "hidden" ); });
+
+// Remove whitelist functionality if the domain was blocked because of a wildcard
+$( "#output" ).bind("append", function(){
+	if($( "#output" ).contents()[0].data.indexOf("Wildcard blocking") !== -1)
+	{
+		$( "#whitelisting" ).hide();
+		$( "#whitelistingform" ).hide();
+	}
+});
+
+function add() {
+	var domain = $("#domain");
+	var pw = $("#pw");
+	if(domain.val().length === 0){
+		return;
+	}
+
+	$.ajax({
+		url: "admin/scripts/pi-hole/php/add.php",
+		method: "post",
+		data: {"domain":domain.val(), "list":"white", "pw":pw.val()},
+		success: function(response) {
+			$( "#whitelistingoutput" ).removeAttr( "hidden" );
+			if(response.indexOf("Pi-hole blocking") !== -1)
+			{
+				// Reload page after 5 seconds
+				setTimeout(function(){window.location.reload(1);}, 5000);
+				$( "#whitelistingoutput" ).html("---> Success <---<br/>You may have to flush your DNS cache");
+			}
+			else
+			{
+				$( "#whitelistingoutput" ).html("---> "+response+" <---");
+			}
+
+		},
+		error: function(jqXHR, exception) {
+			$( "#whitelistingoutput" ).removeAttr( "hidden" );
+			$( "#whitelistingoutput" ).html("---> Unknown Error <---");
+		}
+	});
+}
+// Handle enter button for adding domains
+$(document).keypress(function(e) {
+    if(e.which === 13 && $("#pw").is(":focus")) {
+        add();
+    }
+});
+
+// Handle buttons
+$("#btnAdd").on("click", function() {
+    add();
+});
+</script>
+</body>
+</html>

advanced/lighttpd.conf.debian

@@ -21,7 +21,7 @@ server.modules = (
 )
 
 server.document-root        = "/var/www/html"
-server.error-handler-404	= "pihole/index.html"
+server.error-handler-404	= "pihole/index.php"
 server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
 server.errorlog             = "/var/log/lighttpd/error.log"
 server.pid-file             = "/var/run/lighttpd.pid"

advanced/lighttpd.conf.fedora

@@ -22,7 +22,7 @@ server.modules = (
 )
 
 server.document-root        = "/var/www/html"
-server.error-handler-404	= "pihole/index.html"
+server.error-handler-404	= "pihole/index.php"
 server.upload-dirs          = ( "/var/cache/lighttpd/uploads" )
 server.errorlog             = "/var/log/lighttpd/error.log"
 server.pid-file             = "/var/run/lighttpd.pid"

advanced/logrotate

@@ -0,0 +1,10 @@
+/var/log/pihole.log {
+	# su #
+	daily
+	copytruncate
+	rotate 5
+	compress
+	delaycompress
+	notifempty
+	nomail
+}

advanced/pihole.cron

@@ -8,7 +8,7 @@
 # it under the terms of the GNU General Public License as published by
 # the Free Software Foundation, either version 2 of the License, or
 # (at your option) any later version.
-# 
+#
 # This file is under source-control of the Pi-hole installation and update
 # scripts, any changes made to this file will be overwritten when the softare
 # is updated or re-installed. Please make any changes to the appropriate crontab
@@ -16,11 +16,14 @@
 
 # Pi-hole: Update the ad sources once a week on Sunday at 01:59
 #          Download any updates from the adlists
-59 1    * * 7   root    /usr/local/bin/pihole updateGravity
+59 1    * * 7   root    PATH="$PATH:/usr/local/bin/" pihole updateGravity
 
 # Pi-hole: Update Pi-hole! Uncomment to enable auto update
-#30 2    * * 7   root    /usr/local/bin/pihole updatePihole
+#30 2    * * 7   root    PATH="$PATH:/usr/local/bin/" pihole updatePihole
 
 # Pi-hole: Flush the log daily at 00:00 so it doesn't get out of control
 #          Stats will be viewable in the Web interface thanks to the cron job above
-00 00   * * *   root    /usr/local/bin/pihole flush
+#          The flush script will use logrotate if available
+00 00   * * *   root    PATH="$PATH:/usr/local/bin/" pihole flush
+
+@reboot root /usr/sbin/logrotate /etc/pihole/logrotate

advanced/pihole.sudo

@@ -9,4 +9,3 @@
 # the Free Software Foundation, either version 2 of the License, or
 # (at your option) any later version.
 
-www-data ALL=NOPASSWD: /usr/local/bin/pihole

advanced/selinux/pihole.te

@@ -1,87 +0,0 @@
-module pihole 1.0;
-
-require {
-        type var_log_t;
-        type unconfined_t;
-        type init_t;
-        type auditd_t;
-        type syslogd_t;
-        type NetworkManager_t;
-        type mdadm_t;
-        type tuned_t;
-        type avahi_t;
-        type irqbalance_t;
-        type system_dbusd_t;
-        type kernel_t;
-        type httpd_sys_script_t;
-        type systemd_logind_t;
-        type httpd_t;
-        type policykit_t;
-        type dnsmasq_t;
-        type udev_t;
-        type postfix_pickup_t;
-        type sshd_t;
-        type crond_t;
-        type getty_t;
-        type lvm_t;
-        type postfix_qmgr_t;
-        type postfix_master_t;
-        class dir { getattr search };
-        class file { read open setattr };
-}
-
-#============= dnsmasq_t ==============
-allow dnsmasq_t var_log_t:file { open setattr };
-
-#============= httpd_t ==============
-allow httpd_t var_log_t:file { read open };
-
-#============= httpd_sys_script_t (class: dir) ==============
-allow httpd_sys_script_t NetworkManager_t:dir { getattr search };
-allow httpd_sys_script_t auditd_t:dir { getattr search };
-allow httpd_sys_script_t avahi_t:dir { getattr search };
-allow httpd_sys_script_t crond_t:dir { getattr search };
-allow httpd_sys_script_t dnsmasq_t:dir { getattr search };
-allow httpd_sys_script_t getty_t:dir { getattr search };
-allow httpd_sys_script_t httpd_t:dir { getattr search };
-allow httpd_sys_script_t init_t:dir { getattr search };
-allow httpd_sys_script_t irqbalance_t:dir { getattr search };
-allow httpd_sys_script_t kernel_t:dir { getattr search };
-allow httpd_sys_script_t lvm_t:dir { getattr search };
-allow httpd_sys_script_t mdadm_t:dir { getattr search };
-allow httpd_sys_script_t policykit_t:dir { getattr search };
-allow httpd_sys_script_t postfix_master_t:dir { getattr search };
-allow httpd_sys_script_t postfix_pickup_t:dir { getattr search };
-allow httpd_sys_script_t postfix_qmgr_t:dir { getattr search };
-allow httpd_sys_script_t sshd_t:dir { getattr search };
-allow httpd_sys_script_t syslogd_t:dir { getattr search };
-allow httpd_sys_script_t system_dbusd_t:dir { getattr search };
-allow httpd_sys_script_t systemd_logind_t:dir { getattr search };
-allow httpd_sys_script_t tuned_t:dir { getattr search };
-allow httpd_sys_script_t udev_t:dir { getattr search };
-allow httpd_sys_script_t unconfined_t:dir { getattr search };
-
-#============= httpd_sys_script_t (class: file) ==============
-allow httpd_sys_script_t NetworkManager_t:file { read open };
-allow httpd_sys_script_t auditd_t:file { read open };
-allow httpd_sys_script_t avahi_t:file { read open };
-allow httpd_sys_script_t crond_t:file { read open };
-allow httpd_sys_script_t dnsmasq_t:file { read open };
-allow httpd_sys_script_t getty_t:file { read open };
-allow httpd_sys_script_t httpd_t:file { read open };
-allow httpd_sys_script_t init_t:file { read open };
-allow httpd_sys_script_t irqbalance_t:file { read open };
-allow httpd_sys_script_t kernel_t:file { read open };
-allow httpd_sys_script_t lvm_t:file { read open };
-allow httpd_sys_script_t mdadm_t:file { read open };
-allow httpd_sys_script_t policykit_t:file { read open };
-allow httpd_sys_script_t postfix_master_t:file { read open };
-allow httpd_sys_script_t postfix_pickup_t:file { read open };
-allow httpd_sys_script_t postfix_qmgr_t:file { read open };
-allow httpd_sys_script_t sshd_t:file { read open };
-allow httpd_sys_script_t syslogd_t:file { read open };
-allow httpd_sys_script_t system_dbusd_t:file { read open };
-allow httpd_sys_script_t systemd_logind_t:file { read open };
-allow httpd_sys_script_t tuned_t:file { read open };
-allow httpd_sys_script_t udev_t:file { read open };
-allow httpd_sys_script_t unconfined_t:file { read open };

automated install/uninstall.sh

@@ -82,7 +82,7 @@ removeAndPurge() {
 				read -rp "::: Do you wish to remove ${i} from your system? [y/n]: " yn
 				case ${yn} in
 					[Yy]* ) printf ":::\tRemoving %s..." "${i}"; ${SUDO} ${PKG_REMOVE} "${i}" &> /dev/null & spinner $!; printf "done!\n"; break;;
-					[Nn]* ) printf ":::\tSkipping %s" "${i}\n"; break;;
+					[Nn]* ) printf ":::\tSkipping %s\n" "${i}"; break;;
 					* ) printf "::: You must answer yes or no!\n";;
 				esac
 			done
@@ -136,7 +136,7 @@ removeNoPurge() {
 	fi
 
 	echo "::: Removing config files and scripts..."
-	package_check ${i} > /dev/null
+	package_check lighttpd > /dev/null
 	if [ $? -eq 1 ]; then
 		${SUDO} rm -rf /etc/lighttpd/ &> /dev/null
 	else
@@ -154,6 +154,12 @@ removeNoPurge() {
 	${SUDO} rm /usr/local/bin/pihole &> /dev/null
 	${SUDO} rm /etc/bash_completion.d/pihole &> /dev/null
 	${SUDO} rm /etc/sudoers.d/pihole &> /dev/null
+	
+	# If the pihole user exists, then remove
+	if id "pihole" >/dev/null 2>&1; then
+        	echo "::: Removing pihole user..."
+		${SUDO} userdel -r pihole
+	fi
 
 	echo ":::"
 	printf "::: Finished removing PiHole from your system. Sorry to see you go!\n"

autotest

@@ -0,0 +1 @@
+py.test -v -f test/

gravity.sh

@@ -23,7 +23,7 @@ helpFunc() {
 :::  -f, --force			Force lists to be downloaded, even if they don't need updating.
 :::  -h, --help				Show this help dialog
 EOM
-	exit 1
+	exit 0
 }
 
 
@@ -32,6 +32,7 @@ adListDefault=/etc/pihole/adlists.default
 whitelistScript="pihole -w"
 whitelistFile=/etc/pihole/whitelist.txt
 blacklistFile=/etc/pihole/blacklist.txt
+readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
 
 #Source the setupVars from install script for the IP
 setupVars=/etc/pihole/setupVars.conf
@@ -44,12 +45,14 @@ else
 fi
 
 #Remove the /* from the end of the IPv4addr.
-IPv4_address=${IPv4_address%/*}
+IPV4_ADDRESS=${IPV4_ADDRESS%/*}
+IPV6_ADDRESS=${IPV6_ADDRESS}
 
 # Variables for various stages of downloading and formatting the list
 basename=pihole
 piholeDir=/etc/${basename}
 adList=${piholeDir}/gravity.list
+localList=${piholeDir}/local.list
 justDomainsExtension=domains
 matterAndLight=${basename}.0.matterandlight.txt
 supernova=${basename}.1.supernova.txt
@@ -74,7 +77,7 @@ gravity_collapse() {
 		#custom file found, use this instead of default
 		echo -n "::: Custom adList file detected. Reading..."
 		sources=()
-		while read -r line; do
+		while IFS= read -r line || [[ -n "$line" ]]; do
 			#Do not read commented out or blank lines
 			if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
 				echo "" > /dev/null
@@ -87,7 +90,7 @@ gravity_collapse() {
 		#no custom file found, use defaults!
 		echo -n "::: No custom adlist file detected, reading from default file..."
 		sources=()
-		while read -r line; do
+		while IFS= read -r line || [[ -n "$line" ]]; do
 			#Do not read commented out or blank lines
 			if [[ ${line} = \#* ]] || [[ ! ${line} ]]; then
 				echo "" > /dev/null
@@ -102,16 +105,30 @@ gravity_collapse() {
 # patternCheck - check to see if curl downloaded any new files.
 gravity_patternCheck() {
 	patternBuffer=$1
-	# check if the patternbuffer is a non-zero length file
-	if [[ -s "${patternBuffer}" ]]; then
-		# Some of the blocklists are copyright, they need to be downloaded
-		# and stored as is. They can be processed for content after they
-		# have been saved.
-		mv "${patternBuffer}" "${saveLocation}"
-		echo " List updated, transport successful!"
+	success=$2
+	error=$3
+	if [ $success = true ]; then
+		# check if download was successful but list has not been modified
+		if [ "${error}" == "304" ]; then
+			echo ":::   No changes detected, transport skipped!"
+		# check if the patternbuffer is a non-zero length file
+		elif [[ -s "${patternBuffer}" ]]; then
+			# Some of the blocklists are copyright, they need to be downloaded
+			# and stored as is. They can be processed for content after they
+			# have been saved.
+			mv "${patternBuffer}" "${saveLocation}"
+			echo ":::   List updated, transport successful!"
+		else
+			# Empty file -> use previously downloaded list
+			echo ":::   Received empty file, using cached one (list not updated!)"
+		fi
 	else
-		# curl didn't download any host files, probably because of the date check
-		echo " No changes detected, transport skipped!"
+		# check if cached list exists
+		if [[ -r "${saveLocation}" ]]; then
+			echo ":::   List download failed, using cached list (list not updated!)"
+		else
+			echo ":::   Download failed and no cached list available (list will not be considered)"
+		fi
 	fi
 }
 
@@ -130,9 +147,27 @@ gravity_transport() {
 	fi
 
 	# Silently curl url
-	curl -s -L ${cmd_ext} ${heisenbergCompensator} -A "${agent}" ${url} > ${patternBuffer}
-	# Check for list updates
-	gravity_patternCheck "${patternBuffer}"
+	err=$(curl -s -L ${cmd_ext} ${heisenbergCompensator} -w %{http_code} -A "${agent}" ${url} -o ${patternBuffer})
+
+	echo " done"
+	# Analyze http response
+	echo -n ":::   Status: "
+	case "$err" in
+		"200"  ) echo "Success (OK)"; success=true;;
+		"304"  ) echo "Not modified"; success=true;;
+		"403"  ) echo "Forbidden"; success=false;;
+		"404"  ) echo "Not found"; success=false;;
+		"408"  ) echo "Time-out"; success=false;;
+		"451"  ) echo "Unavailable For Legal Reasons"; success=false;;
+		"521"  ) echo "Web Server Is Down (Cloudflare)"; success=false;;
+		"522"  ) echo "Connection Timed Out (Cloudflare)"; success=false;;
+		"500"  ) echo "Internal Server Error"; success=false;;
+		*      ) echo "Status $err"; success=false;;
+	esac
+
+	# Process result
+	gravity_patternCheck "${patternBuffer}" ${success} "${err}"
+
 }
 
 # spinup - main gravity function
@@ -179,7 +214,10 @@ gravity_Schwarzchild() {
 	echo -n "::: Aggregating list of domains..."
 	truncate -s 0 ${piholeDir}/${matterAndLight}
 	for i in "${activeDomains[@]}"; do
-		cat "${i}" | tr -d '\r' >> ${piholeDir}/${matterAndLight}
+		# Only assimilate list if it is available (download might have faild permanently)
+		if [[ -r "${i}" ]]; then
+			cat "${i}" | tr -d '\r' >> ${piholeDir}/${matterAndLight}
+		fi
 	done
 	echo " done!"
 }
@@ -189,7 +227,7 @@ gravity_Blacklist() {
 	if [[ -f "${blacklistFile}" ]]; then
 	    numBlacklisted=$(wc -l < "${blacklistFile}")
 	    plural=; [[ "$numBlacklisted" != "1" ]] && plural=s
-	    echo -n "::: BlackListing $numBlacklisted domain${plural}..."
+	    echo -n "::: Blacklisting $numBlacklisted domain${plural}..."
 	    cat ${blacklistFile} >> ${piholeDir}/${eventHorizon}
 	    echo " done!"
 	else
@@ -198,6 +236,21 @@ gravity_Blacklist() {
 
 }
 
+gravity_Wildcard() {
+	# Return number of wildcards in output - don't actually handle wildcards
+	if [[ -f "${wildcardlist}" ]]; then
+	    num=$(grep -c ^ "${wildcardlist}")
+	    if [[ -n "${IPV4_ADDRESS}" && -n "${IPV6_ADDRESS}" ]];then
+	        let num/=2
+	    fi
+	    plural=; [[ "$num" != "1" ]] && plural=s
+	    echo "::: Wildcard blocked domain${plural}: $numBlacklisted"
+	else
+	    echo "::: No wildcards used!"
+	fi
+
+}
+
 gravity_Whitelist() {
     #${piholeDir}/${eventHorizon})
 	echo ":::"
@@ -240,30 +293,42 @@ gravity_unique() {
 
 gravity_hostFormat() {
 	# Format domain list as "192.168.x.x domain.com"
-	echo "::: Formatting domains into a HOSTS file..."
-    # Check vars from setupVars.conf to see if we're using IPv4, IPv6, Or both.
-    if [[ -n "${IPv4_address}" && -n "${IPv6_address}" ]];then
+	echo -n "::: Formatting domains into a HOSTS file..."
 
-        # Both IPv4 and IPv6
-        cat ${piholeDir}/${eventHorizon} | awk -v ipv4addr="$IPv4_address" -v ipv6addr="$IPv6_address" '{sub(/\r$/,""); print ipv4addr" "$0"\n"ipv6addr" "$0}' >> ${piholeDir}/${accretionDisc}
+	if [[ -f /etc/hostname ]]; then
+		hostname=$(</etc/hostname)
+	elif [ -x "$(command -v hostname)" ]; then
+		hostname=$(hostname -f)
+	else
+		echo "::: Error: Unable to determine fully qualified domain name of host"
+	fi
+  # Check vars from setupVars.conf to see if we're using IPv4, IPv6, Or both.
+  if [[ -n "${IPV4_ADDRESS}" && -n "${IPV6_ADDRESS}" ]];then
 
-    elif [[ -n "${IPv4_address}" && -z "${IPv6_address}" ]];then
+      echo -e "${IPV4_ADDRESS} ${hostname}\n${IPV6_ADDRESS} ${hostname}\n${IPV4_ADDRESS} pi.hole\n${IPV6_ADDRESS} pi.hole" > ${localList}
+      # Both IPv4 and IPv6
+      cat ${piholeDir}/${eventHorizon} | awk -v ipv4addr="$IPV4_ADDRESS" -v ipv6addr="$IPV6_ADDRESS" '{sub(/\r$/,""); print ipv4addr" "$0"\n"ipv6addr" "$0}' >> ${piholeDir}/${accretionDisc}
 
-        # Only IPv4
-        cat ${piholeDir}/${eventHorizon} | awk -v ipv4addr="$IPv4_address" '{sub(/\r$/,""); print ipv4addr" "$0}' >> ${piholeDir}/${accretionDisc}
+  elif [[ -n "${IPV4_ADDRESS}" && -z "${IPV6_ADDRESS}" ]];then
 
-    elif [[ -z "${IPv4_address}" && -n "${IPv6_address}" ]];then
+      echo -e "${IPV4_ADDRESS} ${hostname}\n${IPV4_ADDRESS} pi.hole" > ${localList}
+      # Only IPv4
+      cat ${piholeDir}/${eventHorizon} | awk -v ipv4addr="$IPV4_ADDRESS" '{sub(/\r$/,""); print ipv4addr" "$0}' >> ${piholeDir}/${accretionDisc}
 
-        # Only IPv6
-        cat ${piholeDir}/${eventHorizon} | awk -v ipv6addr="$IPv6_address" '{sub(/\r$/,""); print ipv6addr" "$0}' >> ${piholeDir}/${accretionDisc}
+  elif [[ -z "${IPV4_ADDRESS}" && -n "${IPV6_ADDRESS}" ]];then
 
-    elif [[ -z "${IPv4_address}" && -z "${IPv6_address}" ]];then
-        echo "::: No IP Values found! Please run 'pihole -r' and choose reconfigure to restore values"
-        exit 1
-    fi
+      echo -e "${IPV6_ADDRESS} ${hostname}\n${IPV6_ADDRESS} pi.hole" > ${localList}
+      # Only IPv6
+      cat ${piholeDir}/${eventHorizon} | awk -v ipv6addr="$IPV6_ADDRESS" '{sub(/\r$/,""); print ipv6addr" "$0}' >> ${piholeDir}/${accretionDisc}
+
+  elif [[ -z "${IPV4_ADDRESS}" && -z "${IPV6_ADDRESS}" ]];then
+      echo "::: No IP Values found! Please run 'pihole -r' and choose reconfigure to restore values"
+      exit 1
+  fi
 
 	# Copy the file over as /etc/pihole/gravity.list so dnsmasq can use it
 	cp ${piholeDir}/${accretionDisc} ${adList}
+	echo " done!"
 }
 
 # blackbody - remove any remnant files from script processes
@@ -287,8 +352,14 @@ gravity_advanced() {
 	echo -n "::: Formatting list of domains to remove comments...."
 	#awk '($1 !~ /^#/) { if (NF>1) {print $2} else {print $1}}' ${piholeDir}/${matterAndLight} | sed -nr -e 's/\.{2,}/./g' -e '/\./p' >  ${piholeDir}/${supernova}
 	#Above line does not correctly grab domains where comment is on the same line (e.g 'addomain.com #comment')
-	#Add additional awk command to read all lines up to a '#', and then continue as we were
-	cat ${piholeDir}/${matterAndLight} | awk -F'#' '{print $1}' | awk '($1 !~ /^#/) { if (NF>1) {print $2} else {print $1}}' | sed -nr -e 's/\.{2,}/./g' -e '/\./p' >  ${piholeDir}/${supernova}
+	#Awk -F splits on given IFS, we grab the right hand side (chops trailing #coments and /'s to grab the domain only.
+	#Last awk command takes non-commented lines and if they have 2 fields, take the left field (the domain) and leave
+	#+ the right (IP address), otherwise grab the single field.
+	cat ${piholeDir}/${matterAndLight} | \
+	    awk -F '#' '{print $1}' | \
+	    awk -F '/' '{print $1}' | \
+	    awk '($1 !~ /^#/) { if (NF>1) {print $2} else {print $1}}' | \
+	    sed -nr -e 's/\.{2,}/./g' -e '/\./p' >  ${piholeDir}/${supernova}
 	echo " done!"
 
 	numberOf=$(wc -l < ${piholeDir}/${supernova})
@@ -306,7 +377,7 @@ gravity_reload() {
 
 	# Reload hosts file
 	echo ":::"
-	echo "::: Refresh lists in dnsmasq..."
+	echo -n "::: Refresh lists in dnsmasq..."
 
 	#ensure /etc/dnsmasq.d/01-pihole.conf is pointing at the correct list!
 	#First escape forward slashes in the path:
@@ -314,7 +385,8 @@ gravity_reload() {
 	#Now replace the line in dnsmasq file
 #	sed -i "s/^addn-hosts.*/addn-hosts=$adList/" /etc/dnsmasq.d/01-pihole.conf
 
-        pihole restartdns
+	pihole restartdns
+	echo " done!"
 }
 
 for var in "$@"; do
@@ -345,6 +417,7 @@ else
 fi
 gravity_Whitelist
 gravity_Blacklist
+gravity_Wildcard
 
 gravity_hostFormat
 gravity_blackbody

pihole

@@ -10,87 +10,142 @@
 # the Free Software Foundation, either version 2 of the License, or
 # (at your option) any later version.
 
+PI_HOLE_SCRIPT_DIR="/opt/pihole"
+readonly wildcardlist="/etc/dnsmasq.d/03-pihole-wildcard.conf"
 # Must be root to use this tool
 if [[ ! $EUID -eq 0 ]];then
-	if [ -x "$(command -v sudo)" ];then
-		exec sudo bash "$0" "$@"
-		exit $?
-	else
-		echo "::: sudo is needed to run pihole commands.  Please run this script as root or install sudo."
-		exit 1
-	fi
+  if [ -x "$(command -v sudo)" ];then
+    exec sudo bash "$0" "$@"
+    exit $?
+  else
+    echo "::: sudo is needed to run pihole commands.  Please run this script as root or install sudo."
+    exit 1
+  fi
 fi
 
+webpageFunc() {
+  source /opt/pihole/webpage.sh
+  main "$@"
+  exit 0
+}
+
 whitelistFunc() {
-	/opt/pihole/list.sh "$@"
-	exit 0
+ "${PI_HOLE_SCRIPT_DIR}"/list.sh "$@"
+  exit 0
 }
 
 blacklistFunc() {
-	/opt/pihole/list.sh "$@"
-	exit 0
+  "${PI_HOLE_SCRIPT_DIR}"/list.sh "$@"
+  exit 0
+}
+
+wildcardFunc() {
+  "${PI_HOLE_SCRIPT_DIR}"/list.sh "$@"
+  exit 0
 }
 
 debugFunc() {
-	/opt/pihole/piholeDebug.sh
-	exit 0
+  "${PI_HOLE_SCRIPT_DIR}"/piholeDebug.sh
+  exit 0
 }
 
 flushFunc() {
-	/opt/pihole/piholeLogFlush.sh
-	exit 0
+  "${PI_HOLE_SCRIPT_DIR}"/piholeLogFlush.sh
+  exit 0
 }
 
-
 updatePiholeFunc() {
-	/opt/pihole/update.sh
-	exit 0
+  "${PI_HOLE_SCRIPT_DIR}"/update.sh
+  exit 0
 }
 
 reconfigurePiholeFunc() {
-	/etc/.pihole/automated\ install/basic-install.sh --reconfigure
-	exit 0;
+  /etc/.pihole/automated\ install/basic-install.sh --reconfigure
+  exit 0;
 }
 
 updateGravityFunc() {
-	/opt/pihole/gravity.sh "$@"
-	exit 0
+  "${PI_HOLE_SCRIPT_DIR}"/gravity.sh "$@"
+  exit 0
 }
 
-setupLCDFunction() {
-	/opt/pihole/setupLCD.sh
-	exit 0
+scanList(){
+  domain="${1}"
+  list="${2}"
+  method="${3}"
+  if [[ ${method} == "-exact" ]] ; then
+    grep -i -E "(^|\s)${domain}($|\s)" "${list}"
+  else
+    grep -i "${domain}" "${list}"
+  fi
+}
+
+processWildcards() {
+  IFS="." read -r -a array <<< "${1}"
+  for (( i=${#array[@]}-1; i>=0; i-- )); do
+    ar=""
+    for (( j=${#array[@]}-1; j>${#array[@]}-i-2; j-- )); do
+      if [[ $j == $((${#array[@]}-1)) ]]; then
+        ar="${array[$j]}"
+      else
+        ar="${array[$j]}.${ar}"
+      fi
+    done
+    echo "${ar}"
+  done
 }
 
 queryFunc() {
-	domain=$2
-	for list in /etc/pihole/list.*; do
-		count=$(grep ${domain} $list | wc -l)
-		echo "::: ${list} (${count} results)"
-			if [[ ${count} > 0 ]]; then
-				grep ${domain} ${list}
-			fi
-		echo ""
-	done
-	exit 0
+  domain="${2}"
+  method="${3}"
+  lists=( /etc/pihole/list.* /etc/pihole/blacklist.txt)
+  for list in ${lists[@]}; do
+    if [ -e "${list}" ]; then
+      result=$(scanList ${domain} ${list} ${method})
+      # Remove empty lines before couting number of results
+      count=$(sed '/^\s*$/d' <<< "$result" | wc -l)
+      echo "::: ${list} (${count} results)"
+      if [[ ${count} > 0 ]]; then
+        echo "${result}"
+      fi
+      echo ""
+    else
+      echo "::: ${list} does not exist"
+      echo ""
+    fi
+  done
+
+  # Scan for possible wildcard matches
+  local wildcards=($(processWildcards "${domain}"))
+  for domain in ${wildcards[@]}; do
+    result=$(scanList "\/${domain}\/" ${wildcardlist})
+    # Remove empty lines before couting number of results
+    count=$(sed '/^\s*$/d' <<< "$result" | wc -l)
+    if [[ ${count} > 0 ]]; then
+      echo "::: Wildcard blocking ${domain} (${count} results)"
+      echo "${result}"
+      echo ""
+    fi
+  done
+  exit 0
 }
 
 chronometerFunc() {
-	shift
-	/opt/pihole/chronometer.sh "$@"
-	exit 0
+  shift
+  "${PI_HOLE_SCRIPT_DIR}"/chronometer.sh "$@"
+  exit 0
 }
 
 
 uninstallFunc() {
-	/opt/pihole/uninstall.sh
-	exit 0
+  "${PI_HOLE_SCRIPT_DIR}"/uninstall.sh
+  exit 0
 }
 
 versionFunc() {
-	shift
-	/opt/pihole/version.sh "$@"
-	exit 0
+  shift
+  "${PI_HOLE_SCRIPT_DIR}"/version.sh "$@"
+  exit 0
 }
 
 restartDNS() {
@@ -115,8 +170,26 @@ restartDNS() {
 piholeEnable() {
   if [[ "${1}" == "0" ]] ; then
     #Disable Pihole
-    sed -i 's/^addn-hosts/#addn-hosts/' /etc/dnsmasq.d/01-pihole.conf
+    sed -i 's/^addn-hosts=\/etc\/pihole\/gravity.list/#addn-hosts=\/etc\/pihole\/gravity.list/' /etc/dnsmasq.d/01-pihole.conf
     echo "::: Blocking has been disabled!"
+    if [[ $# > 1 ]] ; then
+      if [[ ${2} == *"s"* ]] ; then
+        tt=${2%"s"}
+        echo "::: Blocking will be re-enabled in ${tt} seconds"
+        nohup bash -c "sleep ${tt}; pihole enable" </dev/null &>/dev/null &
+      elif [[ ${2} == *"m"* ]] ; then
+        tt=${2%"m"}
+        echo "::: Blocking will be re-enabled in ${tt} minutes"
+        tt=$((${tt}*60))
+        nohup bash -c "sleep ${tt}; pihole enable" </dev/null &>/dev/null &
+      else
+        echo "::: Unknown format for delayed reactivation of the blocking!"
+        echo "::: Example:"
+        echo ":::    pihole disable 5s    -    will disable blocking for 5 seconds"
+        echo ":::    pihole disable 7m    -    will disable blocking for 7 minutes"
+        echo "::: Blocking will not automatically be re-enabled!"
+      fi
+    fi
   else
     #Enable pihole
     echo "::: Blocking has been enabled!"
@@ -125,15 +198,49 @@ piholeEnable() {
   restartDNS
 }
 
+piholeLogging() {
+  shift
+
+  if [[ "${1}" == "off" ]] ; then
+    #Disable Logging
+    sed -i 's/^log-queries/#log-queries/' /etc/dnsmasq.d/01-pihole.conf
+    sed -i 's/^QUERY_LOGGING=true/QUERY_LOGGING=false/' /etc/pihole/setupVars.conf
+    pihole -f
+    echo "::: Logging has been disabled!"
+  elif [[ "${1}" == "on" ]] ; then
+    #Enable logging
+    sed -i 's/^#log-queries/log-queries/' /etc/dnsmasq.d/01-pihole.conf
+    sed -i 's/^QUERY_LOGGING=false/QUERY_LOGGING=true/' /etc/pihole/setupVars.conf
+    echo "::: Logging has been enabled!"
+  else
+    echo "::: Invalid option passed, please pass 'on' or 'off'"
+    exit 1
+  fi
+  restartDNS
+}
+
 piholeStatus() {
-  if [[ $(cat /etc/dnsmasq.d/01-pihole.conf | grep "#addn-hosts=/") ]] ; then
+  if [[ $(netstat -plnt | grep -c ':53 ') > 0 ]]; then
+    if [[ "${1}" != "web" ]] ; then
+      echo "::: DNS service is running"
+    fi
+  else
+    if [[ "${1}" == "web" ]] ; then
+      echo "-1";
+    else
+      echo "::: DNS service is NOT running"
+    fi
+    return
+  fi
+
+  if [[ $(grep -i "^#addn-hosts=/" /etc/dnsmasq.d/01-pihole.conf) ]] ; then
     #list is commented out
     if [[ "${1}" == "web" ]] ; then
       echo 0;
     else
       echo "::: Pi-hole blocking is Disabled";
     fi
-  elif [[ $(cat /etc/dnsmasq.d/01-pihole.conf | grep "addn-hosts=/") ]] ; then
+  elif [[ $(grep -i "^addn-hosts=/" /etc/dnsmasq.d/01-pihole.conf) ]] ; then
     #list set
     if [[ "${1}" == "web" ]] ; then
       echo 1;
@@ -153,57 +260,71 @@ piholeStatus() {
   fi
 }
 
+tailFunc() {
+  echo "Press Ctrl-C to exit"
+  tail -F /var/log/pihole.log
+  exit 0
+}
 
 helpFunc() {
 	cat << EOM
 ::: Control all PiHole specific functions!
 :::
 ::: Usage: pihole [options]
-:::		Add -h after -w (whitelist), -b (blacklist), or -c (chronometer)  for more information on usage
+:::		Add -h after -w (whitelist), -b (blacklist), -c (chronometer), or -a (admin)  for more information on usage
 :::
 ::: Options:
 :::  -w, whitelist            Whitelist domains
 :::  -b, blacklist            Blacklist domains
 :::  -d, debug                Start a debugging session if having trouble
 :::  -f, flush                Flush the pihole.log file
+:::  -t, tail                 Output the last lines of the pihole.log file. Lines are appended as the file grows
 :::  -up, updatePihole        Update Pi-hole
+:::  -r, reconfigure          Reconfigure or Repair Pi-hole
 :::  -g, updateGravity        Update the list of ad-serving domains
-:::  -s, setupLCD             Automatically configures the Pi to use the 2.8 LCD screen to display stats on it
 :::  -c, chronometer          Calculates stats and displays to an LCD
 :::  -h, help                 Show this help dialog
 :::  -v, version              Show current versions
 :::  -q, query                Query the adlists for a specific domain
+:::                           Use pihole -q domain -exact if you want to see exact matches only
+:::  -l, logging              Enable or Disable logging (pass 'on' or 'off')
+:::  -a, admin                Admin webpage options
 :::  uninstall                Uninstall Pi-Hole from your system :(!
 :::  status                   Is Pi-Hole Enabled or Disabled
 :::  enable                   Enable Pi-Hole DNS Blocking
 :::  disable                  Disable Pi-Hole DNS Blocking
+:::                           Blocking can also be disabled only temporarily, e.g.,
+:::                           pihole disable 5m - will disable blocking for 5 minutes
 :::  restartdns               Restart dnsmasq
 EOM
-	exit 1
+  exit 0
 }
 
 if [[ $# = 0 ]]; then
-	helpFunc
+  helpFunc
 fi
 
 # Handle redirecting to specific functions based on arguments
 case "${1}" in
-	"-w" | "whitelist"			) whitelistFunc "$@";;
-	"-b" | "blacklist"			) blacklistFunc "$@";;
-	"-d" | "debug"				) debugFunc;;
-	"-f" | "flush"				) flushFunc;;
-	"-up" | "updatePihole"      ) updatePiholeFunc;;
-	"-r"  | "reconfigure"       ) reconfigurePiholeFunc;;
-	"-g" | "updateGravity"		) updateGravityFunc "$@";;
-	"-s" | "setupLCD"			) setupLCDFunction;;
-	"-c" | "chronometer"		) chronometerFunc "$@";;
-	"-h" | "help"				) helpFunc;;
-	"-v" | "version"            ) versionFunc "$@";;
-	"-q" | "query"              ) queryFunc "$@";;
-	"uninstall"					) uninstallFunc;;
-	"enable"                      ) piholeEnable 1;;
-    "disable"                     ) piholeEnable 0;;
-    "status"                      ) piholeStatus "$2";;
-    "restartdns"                  ) restartDNS;;
-	*                    		) helpFunc;;
+  "-w" | "whitelist"            ) whitelistFunc "$@";;
+  "-b" | "blacklist"            ) blacklistFunc "$@";;
+  "-wild" | "wildcard"          ) wildcardFunc "$@";;
+  "-d" | "debug"                ) debugFunc;;
+  "-f" | "flush"                ) flushFunc;;
+  "-up" | "updatePihole"        ) updatePiholeFunc;;
+  "-r"  | "reconfigure"         ) reconfigurePiholeFunc;;
+  "-g" | "updateGravity"        ) updateGravityFunc "$@";;
+  "-c" | "chronometer"          ) chronometerFunc "$@";;
+  "-h" | "help"                 ) helpFunc;;
+  "-v" | "version"              ) versionFunc "$@";;
+  "-q" | "query"                ) queryFunc "$@";;
+  "-l" | "logging"              ) piholeLogging "$@";;
+  "uninstall"                   ) uninstallFunc;;
+  "enable"                      ) piholeEnable 1;;
+  "disable"                     ) piholeEnable 0 $2;;
+  "status"                      ) piholeStatus "$2";;
+  "restartdns"                  ) restartDNS;;
+  "-a" | "admin"                ) webpageFunc "$@";;
+  "-t" | "tail"                 ) tailFunc;;
+  *                             ) helpFunc;;
 esac

requirements.txt

@@ -0,0 +1,5 @@
+docker-compose
+pytest
+pytest-xdist
+pytest-cov
+testinfra

test/centos.Dockerfile

@@ -0,0 +1,16 @@
+FROM centos:7
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/conftest.py

@@ -0,0 +1,61 @@
+import pytest
+import testinfra
+
+check_output = testinfra.get_backend(
+    "local://"
+).get_module("Command").check_output
+
+@pytest.fixture
+def Pihole(Docker):
+    ''' used to contain some script stubbing, now pretty much an alias.
+    Also provides bash as the default run function shell '''
+    def run_bash(self, command, *args, **kwargs):
+        cmd = self.get_command(command, *args)
+        if self.user is not None:
+            out = self.run_local(
+                "docker exec -u %s %s /bin/bash -c %s",
+                self.user, self.name, cmd)
+        else:
+            out = self.run_local(
+                "docker exec %s /bin/bash -c %s", self.name, cmd)
+        out.command = self.encode(cmd)
+        return out
+
+    funcType = type(Docker.run)
+    Docker.run = funcType(run_bash, Docker, testinfra.backend.docker.DockerBackend)
+    return Docker
+
+@pytest.fixture
+def Docker(request, args, image, cmd):
+    ''' combine our fixtures into a docker run command and setup finalizer to cleanup '''
+    assert 'docker' in check_output('id'), "Are you in the docker group?"
+    docker_run = "docker run {} {} {}".format(args, image, cmd)
+    docker_id = check_output(docker_run)
+
+    def teardown():
+        check_output("docker rm -f %s", docker_id)
+    request.addfinalizer(teardown)
+
+    docker_container = testinfra.get_backend("docker://" + docker_id)
+    docker_container.id = docker_id
+    return docker_container
+
+@pytest.fixture
+def args(request):
+    ''' -t became required when tput began being used '''
+    return '-t -d'
+
+@pytest.fixture(params=['debian', 'centos'])
+def tag(request):
+    ''' consumed by image to make the test matrix '''
+    return request.param
+
+@pytest.fixture()
+def image(request, tag):
+    ''' built by test_000_build_containers.py '''
+    return 'pytest_pihole:{}'.format(tag)
+
+@pytest.fixture()
+def cmd(request):
+    ''' default to doing nothing by tailing null, but don't exit '''
+    return 'tail -f /dev/null'

test/debian.Dockerfile

@@ -0,0 +1,16 @@
+FROM debian:jessie
+
+ENV GITDIR /etc/.pihole
+ENV SCRIPTDIR /opt/pihole
+
+RUN mkdir -p $GITDIR $SCRIPTDIR /etc/pihole
+ADD . $GITDIR
+RUN cp $GITDIR/advanced/Scripts/*.sh $GITDIR/gravity.sh $GITDIR/pihole $GITDIR/automated\ install/*.sh $SCRIPTDIR/
+ENV PATH /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:$SCRIPTDIR
+
+RUN true && \
+    chmod +x $SCRIPTDIR/*
+
+ENV PH_TEST true
+
+#sed '/# Start the installer/Q' /opt/pihole/basic-install.sh > /opt/pihole/stub_basic-install.sh && \

test/test_000_build_containers.py

@@ -0,0 +1,18 @@
+''' This file starts with 000 to make it run first '''
+import pytest
+import testinfra
+
+run_local = testinfra.get_backend(
+    "local://"
+).get_module("Command").run
+
+@pytest.mark.parametrize("image,tag", [
+    ( 'test/debian.Dockerfile', 'pytest_pihole:debian' ),
+    ( 'test/centos.Dockerfile', 'pytest_pihole:centos' ),
+])
+def test_build_pihole_image(image, tag):
+    build_cmd = run_local('docker build -f {} -t {} .'.format(image, tag))
+    if build_cmd.rc != 0:
+        print build_cmd.stdout
+        print build_cmd.stderr
+    assert build_cmd.rc == 0

test/test_automated_install.py

@@ -0,0 +1,302 @@
+import pytest
+from textwrap import dedent
+
+SETUPVARS = {
+    'PIHOLE_INTERFACE' : 'eth99',
+    'IPV4_ADDRESS' : '1.1.1.1',
+    'IPV6_ADDRESS' : 'FE80::240:D0FF:FE48:4672',
+    'PIHOLE_DNS_1' : '4.2.2.1',
+    'PIHOLE_DNS_2' : '4.2.2.2'
+}
+
+def test_setupVars_are_sourced_to_global_scope(Pihole):
+    ''' currently update_dialogs sources setupVars with a dot,
+    then various other functions use the variables.
+    This confirms the sourced variables are in scope between functions '''
+    setup_var_file = 'cat <<EOF> /etc/pihole/setupVars.conf\n'
+    for k,v in SETUPVARS.iteritems():
+        setup_var_file += "{}={}\n".format(k, v)
+    setup_var_file += "EOF\n"
+    Pihole.run(setup_var_file)
+
+    script = dedent('''\
+    set -e
+    printSetupVars() {
+        # Currently debug test function only
+        echo "Outputting sourced variables"
+        echo "PIHOLE_INTERFACE=${PIHOLE_INTERFACE}"
+        echo "IPV4_ADDRESS=${IPV4_ADDRESS}"
+        echo "IPV6_ADDRESS=${IPV6_ADDRESS}"
+        echo "PIHOLE_DNS_1=${PIHOLE_DNS_1}"
+        echo "PIHOLE_DNS_2=${PIHOLE_DNS_2}"
+    }
+    update_dialogs() {
+        . /etc/pihole/setupVars.conf
+    }
+    update_dialogs
+    printSetupVars
+    ''')
+
+    output = run_script(Pihole, script).stdout
+
+    for k,v in SETUPVARS.iteritems():
+        assert "{}={}".format(k, v) in output
+
+def test_setupVars_saved_to_file(Pihole):
+    ''' confirm saved settings are written to a file for future updates to re-use '''
+    set_setup_vars = '\n'  # dedent works better with this and padding matching script below
+    for k,v in SETUPVARS.iteritems():
+        set_setup_vars += "    {}={}\n".format(k, v)
+    Pihole.run(set_setup_vars).stdout
+
+    script = dedent('''\
+    set -e
+    echo start
+    TERM=xterm
+    source /opt/pihole/basic-install.sh
+    {}
+    finalExports
+    cat /etc/pihole/setupVars.conf
+    '''.format(set_setup_vars))
+
+    output = run_script(Pihole, script).stdout
+
+    for k,v in SETUPVARS.iteritems():
+        assert "{}={}".format(k, v) in output
+
+def test_configureFirewall_firewalld_running_no_errors(Pihole):
+    ''' confirms firewalld rules are applied when firewallD is running '''
+    # firewallD returns 'running' as status
+    mock_command('firewall-cmd', {'*':('running', 0)}, Pihole)
+    # Whiptail dialog returns Ok for user prompt
+    mock_command('whiptail', {'*':('', 0)}, Pihole)
+    configureFirewall = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    configureFirewall
+    ''')
+    expected_stdout = 'Configuring FirewallD for httpd and dnsmasq.'
+    assert expected_stdout in configureFirewall.stdout
+    firewall_calls = Pihole.run('cat /var/log/firewall-cmd').stdout
+    assert 'firewall-cmd --state' in firewall_calls
+    assert 'firewall-cmd --permanent --add-port=80/tcp --add-port=53/tcp --add-port=53/udp' in firewall_calls
+    assert 'firewall-cmd --reload' in firewall_calls
+
+def test_configureFirewall_firewalld_disabled_no_errors(Pihole):
+    ''' confirms firewalld rules are not applied when firewallD is not running '''
+    # firewallD returns non-running status
+    mock_command('firewall-cmd', {'*':('not running', '1')}, Pihole)
+    configureFirewall = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    configureFirewall
+    ''')
+    expected_stdout = 'No active firewall detected.. skipping firewall configuration.'
+    assert expected_stdout in configureFirewall.stdout
+
+def test_configureFirewall_firewalld_enabled_declined_no_errors(Pihole):
+    ''' confirms firewalld rules are not applied when firewallD is running, user declines ruleset '''
+    # firewallD returns running status
+    mock_command('firewall-cmd', {'*':('running', 0)}, Pihole)
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*':('', 1)}, Pihole)
+    configureFirewall = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    configureFirewall
+    ''')
+    expected_stdout = 'Not installing firewall rulesets.'
+    assert expected_stdout in configureFirewall.stdout
+
+def test_configureFirewall_no_firewall(Pihole):
+    ''' confirms firewall skipped no daemon is running '''
+    configureFirewall = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    configureFirewall
+    ''')
+    expected_stdout = 'No active firewall detected'
+    assert expected_stdout in configureFirewall.stdout
+
+def test_configureFirewall_IPTables_enabled_declined_no_errors(Pihole):
+    ''' confirms IPTables rules are not applied when IPTables is running, user declines ruleset '''
+    # iptables command exists
+    mock_command('iptables', {'*':('', '0')}, Pihole)
+    # modinfo returns always true (ip_tables module check)
+    mock_command('modinfo', {'*':('', '0')}, Pihole)
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*':('', '1')}, Pihole)
+    configureFirewall = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    configureFirewall
+    ''')
+    expected_stdout = 'Not installing firewall rulesets.'
+    assert expected_stdout in configureFirewall.stdout
+
+def test_configureFirewall_IPTables_enabled_rules_exist_no_errors(Pihole):
+    ''' confirms IPTables rules are not applied when IPTables is running and rules exist '''
+    # iptables command exists and returns 0 on calls (should return 0 on iptables -C)
+    mock_command('iptables', {'-S':('-P INPUT DENY', '0')}, Pihole)
+    # modinfo returns always true (ip_tables module check)
+    mock_command('modinfo', {'*':('', '0')}, Pihole)
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*':('', '0')}, Pihole)
+    configureFirewall = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    configureFirewall
+    ''')
+    expected_stdout = 'Installing new IPTables firewall rulesets'
+    assert expected_stdout in configureFirewall.stdout
+    firewall_calls = Pihole.run('cat /var/log/iptables').stdout
+    assert 'iptables -I INPUT 1 -p tcp -m tcp --dport 80 -j ACCEPT' not in firewall_calls
+    assert 'iptables -I INPUT 1 -p tcp -m tcp --dport 53 -j ACCEPT' not in firewall_calls
+    assert 'iptables -I INPUT 1 -p udp -m udp --dport 53 -j ACCEPT' not in firewall_calls
+
+def test_configureFirewall_IPTables_enabled_not_exist_no_errors(Pihole):
+    ''' confirms IPTables rules are applied when IPTables is running and rules do not exist '''
+    # iptables command and returns 0 on calls (should return 1 on iptables -C)
+    mock_command('iptables', {'-S':('-P INPUT DENY', '0'), '-C':('', 1), '-I':('', 0)}, Pihole)
+    # modinfo returns always true (ip_tables module check)
+    mock_command('modinfo', {'*':('', '0')}, Pihole)
+    # Whiptail dialog returns Cancel for user prompt
+    mock_command('whiptail', {'*':('', '0')}, Pihole)
+    configureFirewall = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    configureFirewall
+    ''')
+    expected_stdout = 'Installing new IPTables firewall rulesets'
+    assert expected_stdout in configureFirewall.stdout
+    firewall_calls = Pihole.run('cat /var/log/iptables').stdout
+    assert 'iptables -I INPUT 1 -p tcp -m tcp --dport 80 -j ACCEPT' in firewall_calls
+    assert 'iptables -I INPUT 1 -p tcp -m tcp --dport 53 -j ACCEPT' in firewall_calls
+    assert 'iptables -I INPUT 1 -p udp -m udp --dport 53 -j ACCEPT' in firewall_calls
+
+def test_installPiholeWeb_fresh_install_no_errors(Pihole):
+    ''' confirms all web page assets from Core repo are installed on a fresh build '''
+    installWeb = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    installPiholeWeb
+    ''')
+    assert 'Installing pihole custom index page...' in installWeb.stdout
+    assert 'No default index.lighttpd.html file found... not backing up' in installWeb.stdout
+    web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
+    assert 'index.php' in web_directory
+    assert 'index.js' in web_directory
+    assert 'blockingpage.css' in web_directory
+
+def test_installPiholeWeb_empty_directory_no_errors(Pihole):
+    ''' confirms all web page assets from Core repo are installed in an emtpy directory '''
+    installWeb = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    mkdir -p /var/www/html/pihole
+    installPiholeWeb
+    ''')
+    assert 'Installing pihole custom index page...' in installWeb.stdout
+    assert 'No default index.lighttpd.html file found... not backing up' not in installWeb.stdout
+    assert 'index.php missing, replacing...' in installWeb.stdout
+    assert 'index.js missing, replacing...' in installWeb.stdout
+    assert 'blockingpage.css missing, replacing...' in installWeb.stdout
+    web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
+    assert 'index.php' in web_directory
+    assert 'index.js' in web_directory
+    assert 'blockingpage.css' in web_directory
+
+def test_installPiholeWeb_index_php_no_errors(Pihole):
+    ''' confirms all web page assets from Core repo are installed when necessary '''
+    installWeb = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    mkdir -p /var/www/html/pihole
+    touch /var/www/html/pihole/index.php
+    installPiholeWeb
+    ''')
+    assert 'Installing pihole custom index page...' in installWeb.stdout
+    assert 'No default index.lighttpd.html file found... not backing up' not in installWeb.stdout
+    assert 'Existing index.php detected, not overwriting' in installWeb.stdout
+    assert 'index.js missing, replacing...' in installWeb.stdout
+    assert 'blockingpage.css missing, replacing...' in installWeb.stdout
+    web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
+    assert 'index.php' in web_directory
+    assert 'index.js' in web_directory
+    assert 'blockingpage.css' in web_directory
+
+def test_installPiholeWeb_index_js_no_errors(Pihole):
+    ''' confirms all web page assets from Core repo are installed when necessary '''
+    installWeb = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    mkdir -p /var/www/html/pihole
+    touch /var/www/html/pihole/index.js
+    installPiholeWeb
+    ''')
+    assert 'Installing pihole custom index page...' in installWeb.stdout
+    assert 'No default index.lighttpd.html file found... not backing up' not in installWeb.stdout
+    assert 'index.php missing, replacing...' in installWeb.stdout
+    assert 'Existing index.js detected, not overwriting' in installWeb.stdout
+    assert 'blockingpage.css missing, replacing...' in installWeb.stdout
+    web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
+    assert 'index.php' in web_directory
+    assert 'index.js' in web_directory
+    assert 'blockingpage.css' in web_directory
+
+def test_installPiholeWeb_blockingpage_css_no_errors(Pihole):
+    ''' confirms all web page assets from Core repo are installed when necessary '''
+    installWeb = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    mkdir -p /var/www/html/pihole
+    touch /var/www/html/pihole/blockingpage.css
+    installPiholeWeb
+    ''')
+    assert 'Installing pihole custom index page...' in installWeb.stdout
+    assert 'No default index.lighttpd.html file found... not backing up' not in installWeb.stdout
+    assert 'index.php missing, replacing...' in installWeb.stdout
+    assert 'index.js missing, replacing...' in installWeb.stdout
+    assert 'Existing blockingpage.css detected, not overwriting' in installWeb.stdout
+    web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
+    assert 'index.php' in web_directory
+    assert 'index.js' in web_directory
+    assert 'blockingpage.css' in web_directory
+
+def test_installPiholeWeb_already_populated_no_errors(Pihole):
+    ''' confirms all web page assets from Core repo are installed when necessary '''
+    installWeb = Pihole.run('''
+    source /opt/pihole/basic-install.sh
+    mkdir -p /var/www/html/pihole
+    touch /var/www/html/pihole/index.php
+    touch /var/www/html/pihole/index.js
+    touch /var/www/html/pihole/blockingpage.css
+    installPiholeWeb
+    ''')
+    assert 'Installing pihole custom index page...' in installWeb.stdout
+    assert 'No default index.lighttpd.html file found... not backing up' not in installWeb.stdout
+    assert 'Existing index.php detected, not overwriting' in installWeb.stdout
+    assert 'index.php missing, replacing...' not in installWeb.stdout
+    assert 'Existing index.js detected, not overwriting' in installWeb.stdout
+    assert 'index.js missing, replacing...' not in installWeb.stdout
+    assert 'Existing blockingpage.css detected, not overwriting' in installWeb.stdout
+    assert 'blockingpage.css missing, replacing... ' not in installWeb.stdout
+    web_directory = Pihole.run('ls -r /var/www/html/pihole').stdout
+    assert 'index.php' in web_directory
+    assert 'index.js' in web_directory
+    assert 'blockingpage.css' in web_directory
+
+# Helper functions
+def mock_command(script, args, container):
+    ''' Allows for setup of commands we don't really want to have to run for real in unit tests '''
+    full_script_path = '/usr/local/bin/{}'.format(script)
+    mock_script = dedent('''\
+    #!/bin/bash -e
+    echo "\$0 \$@" >> /var/log/{script}
+    case "\$1" in'''.format(script=script))
+    for k, v in args.iteritems():
+        case = dedent('''
+        {arg})
+        echo {res}
+        exit {retcode}
+        ;;'''.format(arg=k, res=v[0], retcode=v[1]))
+        mock_script += case
+    mock_script += dedent('''
+    esac''')
+    container.run('''
+    cat <<EOF> {script}\n{content}\nEOF
+    chmod +x {script}
+    rm -f /var/log/{scriptlog}'''.format(script=full_script_path, content=mock_script, scriptlog=script))
+
+def run_script(Pihole, script):
+    result = Pihole.run(script)
+    assert result.rc == 0
+    return result

test/test_shellcheck.py

@@ -0,0 +1,13 @@
+import pytest
+import testinfra
+
+run_local = testinfra.get_backend(
+    "local://"
+).get_module("Command").run
+
+def test_scripts_pass_shellcheck():
+    ''' Make sure shellcheck does not find anything wrong with our shell scripts '''
+    shellcheck = "find . -type f \( -name 'update.sh' -o -name 'piholeDebug.sh' \) | while read file; do shellcheck \"$file\"; done;"
+    results = run_local(shellcheck)
+    print results.stdout
+    assert '' == results.stdout