Commit Graph

510 Commits

Author SHA1 Message Date
DL6ER
2c497a9a3e Add protocol validation when downloading blocklist from URL
Signed-off-by: DL6ER <dl6er@dl6er.de>
2024-07-05 18:35:19 +01:00
Orazio
d80fcf2e62 More checks when downloading from file:// scheme
Signed-off-by: Orazio <22700499+orazioedoardo@users.noreply.github.com>
2024-03-27 22:10:12 +01:00
Adam Warner
eb7daf4d2c Fix file permission check in gravity.sh. Remove quotes that were added after complaints from shellcheck, this stopped the comparisson from working
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2024-03-27 19:12:59 +00:00
DL6ER
9dd138b033 Only use local files (file://) when they have explicit permissions a+r
Signed-off-by: DL6ER <dl6er@dl6er.de>
2024-03-04 19:38:13 +01:00
DL6ER
3f7413d538 Add "-ni" to all sqlite3 invocations
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-12-09 23:07:35 +01:00
Christian König
cdc1c1ace1 Remove user agent when downloading adlists
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-09-03 23:40:58 +02:00
DL6ER
6c302c9bc8 Adlist properties need to be stored in the new ("temp") database rather than the old.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2023-05-30 17:01:58 +02:00
DL6ER
73733308ba Use parseList function offered by pihole-FTL 2023-05-15 19:25:56 +02:00
Christian König
eaded9fdb1 Remove forgotten variables
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-05-11 22:16:21 +02:00
Christian König
a3e610dbf2 Don't use '--suffix' in mktemp
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-05-10 06:52:51 +02:00
yubiuser
cd638b102f Filter ABP CSS selectors (#5247) 2023-05-04 12:01:20 +02:00
Christian König
dd3a7a4edb Only delete lines containing separator when preceded by a letter to reduce false positiv (deleting valid comments)
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-08 14:59:26 +02:00
Christian König
d10d59303e There is no ! but an $ rule
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-07 21:44:49 +02:00
Christian König
aaf828117d Remove unecessary $cmd_ext
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-07 12:36:50 +02:00
Christian König
9c4e74ffa7 Remove special handling of pgl.yoyo.org
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-07 12:23:11 +02:00
Christian König
2a0f720153 Don't delete lines containing / as they should count as invalid
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-07 10:25:25 +02:00
Christian König
cc17fe18a9 Remove lines with ABP extended CSS selector
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-07 10:14:59 +02:00
Christian König
61ff5b2c76 Unifiy sed commands
Co-authored-by: RD WebDesign <github@rdwebdesign.com.br>
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-04-07 09:44:31 +02:00
RD WebDesign
66bfa606a7 Using a better text for the comment
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2023-03-29 14:17:41 -03:00
RD WebDesign
32fb2e69ff Spliting the regex into TLD_pattern and subdomain_pattern
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2023-03-29 00:05:01 -03:00
RD WebDesign
c71460e4b6 Allow TLD blocking using ABP style
This validates patterns without dots (only for abp style), allowing TLDs to be blocked

Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2023-03-28 17:21:50 -03:00
Adam Warner
0656ceb149 Speed things up a bit with some humble greps. Consolodate regexes and remove the need for so many mapfile/arrays
Signed-off-by: Adam Warner <me@adamwarner.co.uk>

Use temp files for parsing and remove when done.

Always rm the non-domains temp file.

exit 1 if gravity database creation fails.

Signed-off-by: Dan Schaper <dan.schaper@pi-hole.net>
Co-authored-by: Dan Schaper <dan.schaper@pi-hole.net>
Co-authored-by: DL6ER <dl6er@dl6er.de>
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
2023-03-25 11:19:23 -07:00
kot0dama
76b7453f90 Add configurable GRAVITY_TMPDIR variable into setupVars
Signed-off-by: Loïc Gomez <89980752+kot0dama@users.noreply.github.com>
2023-03-22 19:40:24 +09:00
Christian König
71e262c37f Revert "Allow final dot (root zone)"
This reverts commit 0b5da9f0da.

Signed-off-by: Christian König <ckoenig@posteo.de>
2023-03-14 19:50:53 +01:00
Christian König
0b5da9f0da Allow final dot (root zone)
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-03-04 10:36:07 +01:00
Christian König
16385af3ef Use dedicated pattern variable to make RegEx reusable
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-26 21:16:45 +01:00
Christian König
6cb0be82ca Add flag abp_domains into info table to signal if abp domains have been found
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-26 10:34:17 +01:00
Christian König
821c7dc190 Add info when list cotains ABP style domains
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-24 22:18:41 +01:00
Christian König
eedd93d782 Remove temporary files
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-24 21:55:22 +01:00
Adam Warner
1f3f849106 Remove lines starting with ! or [ to account for ABP style comments and header
Also splits the piped "one-liner" in ParseFileIntoDomains into individually commented commands (makes for easier reading and debugging)

Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2023-02-22 21:21:38 +01:00
Christian König
ca74152d1d Allow adding ABP style blocklists
Co-authored-by: Adam Warner <me@adamwarner.co.uk>
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-22 21:21:13 +01:00
Christian König
a4bdf2454b Don't use intermediate strings to filter false positives in gravity
Signed-off-by: Christian König <ckoenig@posteo.de>
2023-02-16 10:39:21 +01:00
Adam Warner
9939cf1d77 Rename some of the variables to hopefully make the process a little clearer
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2023-01-21 23:57:49 +00:00
Adam Warner
79f4a7cef0 Maintain a list of valid hostnames (taken from the top of StevenBlack's host file), and discount them when calculating the number of "invalid" domains in a given list
Soften the output message when reporting on unsuable domains

Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2023-01-20 18:52:14 +00:00
Christian König
14ab586603 Start counting with 1
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-12-11 15:12:28 +01:00
Christian König
372070ab39 Improve detection of incorrecr lines during gravity
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-12-11 15:12:28 +01:00
Christian König
4004a93d1a Don't double check for lines containing no domains in gravity
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-12-11 15:12:28 +01:00
RD WebDesign
090727b1d3 Remove unnecessary return
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2022-10-17 15:24:49 -03:00
RD WebDesign
8b98fd06ed Remove unused old code
Signed-off-by: RD WebDesign <github@rdwebdesign.com.br>
2022-10-16 15:54:24 -03:00
Christian König
230d6a435f Semi hardcode the connection_timeout value
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-08-31 21:41:57 +02:00
Christian König
53c0982c8b Set connection timeout for curl
Signed-off-by: Christian König <ckoenig@posteo.de>
2022-08-31 14:43:20 +02:00
Adam Warner
64dc002c3f Same per @rdwebdesign suggestion.
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2022-08-01 20:38:31 +01:00
Adam Warner
6c7a28a7b5 Dont use bash keywords/programs as variable names 2022-08-01 18:38:15 +01:00
Christian König
b486786041 Add database healthy checks to debug script
Fix health check output comparison in gravity script as well
Add note about waiting time

Signed-off-by: Christian König <ckoenig@posteo.de>
2022-07-10 23:32:47 +02:00
Adam Warner
0be987239e Adds spellcheck action to the smoke-tests (also moves editorconfig-checker to smoketests away from it's own action)
Signed-off-by: Adam Warner <me@adamwarner.co.uk>
2022-07-02 14:04:08 +01:00
Peter Russell
052a1f294e Update gravity.sh
Co-authored-by: yubiuser <ckoenig@posteo.de>
2022-04-20 10:44:05 +02:00
Peter Russell
325f20d5c2 Update gravity.sh
Co-authored-by: yubiuser <ckoenig@posteo.de>
2022-04-20 10:43:56 +02:00
Peter Russell
1c28da7bbd Update gravity.sh
Co-authored-by: yubiuser <ckoenig@posteo.de>
2022-04-20 10:43:39 +02:00
jpgpi250
aba5e884eb Update gravity.sh
Signed-off-by: jpgpi250 <jpgpi250@gmail.com>
2022-04-20 10:19:44 +02:00
DL6ER
8cbffa179d Replace remaining sqlite3 calls by calls to our embedded pihole-FTL sqlite3 engine and remove sqlite3 as dependency in the installer.
Signed-off-by: DL6ER <dl6er@dl6er.de>
2022-01-30 11:18:17 +01:00