The Adblock Project

Jakobud · Guest

Why don't we do this? We can post all of our filters or even our filter files so everyone can download them and import them into Adblock.

Org · Joined: 23 Oct 2003 Posts: 349

Why not indeed? This is what I'm using right now (works well, but it's still evolving slowly):

rue · Developer Joined: 22 Oct 2003 Posts: 752

Here's a list from early August.
.
It's pretty good for macintosh / major news-sites; duplicates were entered for testing.
.
Oh, and prepended exclamation-marks disable stuff.

Aaron Spuler · QA Testing Joined: 23 Oct 2003 Posts: 19

http://mozthemes.tk -- go to adblock --> filters. that's the list i've compiled since using adblock and it blocks basically everything for me. check it out and see what you think Smile

Henrik · Posted: Sat Nov 01, 2003 Post subject: Warning

The idea of compiling the ultimate filter list is appealing, but may present problems. The reason for this is that Adblock works by matching the address of every blockable object against every filter in the list. In algorithmic terms, this scales as O(M*N).

This is actually not a very effective way of doing it, but it's necessary because of the use of filters instead of just blacklisting certain servers, and as long as the list is kept small most of us will never notice any delays.

What I'm trying to say is that compiling a huge list of every ad-directory on every server on the planet will scale extremely badly. The idea of creating a "beginner's list" is very good, though, but should perhaps be limited to containing very generic filters (*/ads/*, *doubleclick*, etc). At the moment, every filter-list should be adapted to the needs of the individual user and his surfing habits.

It should be noted that one of Rue's "near future"-plans are (correct me if I say something wrong here, Rue!) to make an auto-pruning feature for unused filters. When this arrives, a huge list will be a good idea, as it will slowly addapt itself to the users need as it's used.

Lanny Chambers · Guest

We're all running BannerBlind, right? BB takes a large load off the Adblock filter list.

Now, if only there were a way to block all Flash movies smaller than a certain pixel size...

Org · Joined: 23 Oct 2003 Posts: 349

I used to run Bannerblind in Mozilla before I changed to Firebird. Now with FB and Adblock I see absolutely no need for Bannerblind. Adblock simply blocks everything unwanted.

Guest · Posted: Thu Nov 06, 2003 Post subject:

my general filters:

McMurmel · Joined: 13 Nov 2003 Posts: 19 Location: Germany

Don't kill me but if you want to use regular expressions the string must begin and end with an / ?

so the correct list would look like /[^a-zA-Z]adcycle/ instead of /[^a-zA-Z]adcycle* and so on?
_________________
The road goes ever on and on - down from the door that it began - now far ahead the road has gone and I must follow if I can...

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest:
You can't mix simple and RegExp syntax. Regular Expressions must begin and end with the forward-slash (/) and the wildcard is period+asterisk (.*). That list is mostly wrong.

McMurmel:
Ignore the expressions above you. The example you were correcting should read: /[^a-zA-Z]adcycle.*/

McMurmel · Joined: 13 Nov 2003 Posts: 19 Location: Germany

Sorry to correct you but what is the difference between these three regular expressions?:

1. /[^a-zA-Z]adcycle/
2. /[^a-zA-Z]adcycle.*/
3. /.*[^a-zA-Z]adcycle.*/

As long as we don't use multiline-strings there's no difference (and we don't because url's are single-line-strings). They all deliver the same result. .* means any character but the newline-character (\n) 0 - oo times, so the first is the simplest of all three and should be prefered.

P.S: That the reason I suggested a change in entering simple filters.
_________________
The road goes ever on and on - down from the door that it began - now far ahead the road has gone and I must follow if I can...

rue · Developer Joined: 22 Oct 2003 Posts: 752

McMurmel:
Yea- I knew that, but in the interest of preserving the original author's intent, I converted his wildcard. I didn't realize this nick was yours (Stefan), so I tried to be as unconfusing as possible :P

McMurmel · Joined: 13 Nov 2003 Posts: 19 Location: Germany

Weber · Joined: 16 Nov 2003 Posts: 1

This filter get rid of everything, havnt seen a ad in weeks(after upgrading to the newest adblock... good work)

[AdBlock]
*/ad/*
*/ads/*
*/click/*
*/fastclick/*
*ADV*
*adlog*
*ads*
*adsdk*
*adserver*
*adtech*
*advertising*
*annone*
*atdmt*
*banners*
*click.*
*doubleclick*
*image.ugo.com*
*m2*
*sob*
*spinbox*
*viewad*

rue · Developer Joined: 22 Oct 2003 Posts: 752

The fourth pass of build 17 (posted last evening) allows unanchored simple-expressions.
.
This means you can replace *spinbox* with spinbox -- and it works just fine.

NJH · Joined: 13 Nov 2003 Posts: 183 Location: Hampshire, England

The idea of the ultimate filter list is not bad, but I do not think the wheel should be totally re-invented. This is not the only ad blocking software using regular expressions in its block file. Have a look at this thread I started as a guest. Rue very kindly converted the list into the correct format for AdBlock as the original used a slightly different dialect of regular expressions. It is a fairly large list and I have no idea what performance hit a PC would take when using it. I have not timed it. There may well be other lists already available on the web.

One thing which has to be watched is oversimplification of the block file as you possibly see on Weber's list. If my understanding is correct, *ads* would block www.dadsuk.co.uk, www.dadsaregreat.com and sites with things like madscientist or ADSL in the URL etc; *ADV* would block URLs with advice, advantage and so on.

Org · Joined: 23 Oct 2003 Posts: 349

Guest · Posted: Wed Dec 10, 2003 Post subject: one more

/http://ad\d*\./

blocks things like

http://ad.server.com/blabla
http://ad1.server.com/blabla
http://ad42.server.com/blabla

and so on.

rue · Developer Joined: 22 Oct 2003 Posts: 752

I suppose it couldn't hurt -- some further refinements, lifted straight from my list:

/[\W\d]ad(server|s)?[\W\d]/ -- catches .ad., /ad., cgi?ads=19, etc.
/[\W\d]banner(s|id\=)[\W\d]/ -- general banner-stuff
/\D\d{2,3}x\d{2,3}\D/ -- catches banner-sizes in source: 192x74
http://babelfish.altavista.com/babelfish/urltrtop -- removes babelfish's annoying upper-frame

NJH · Joined: 13 Nov 2003 Posts: 183 Location: Hampshire, England

Try something like /[^a-z]ad(s?|v)[^a-z]/. It gets rid of the words ad, ads, and adv bounded by any non-letter which includes . / and any number. This means it also gets rid of things like ad0.
A fuller filter is /[^a-z]ad(s?|serv(er|e)?|v)[^a-z]/

For banners, I use 2 filters to pick up words beginning in banner or ending in banner(s):
/[^a-z]banner/
/banners?[^a-z]/

From previous correspondance, I think your [\W\d] is broadly equivalent to my [^a-z]

rue · Developer Joined: 22 Oct 2003 Posts: 752

NJH:
Using [\W\d] is not only easier on the eyes, but also independant of Adblock's case-insensitivity.
.
Also, you made some prior mistakes again: /[^a-z]ad(s?|serv(er|e)?|v)[^a-z]/
First, an OR-set shouldn't contain zero-match cases if anything follows the set. In other words, (s?|..)[^a-z] should be (s|..)?[^a-z]. Second, you've ordered the set such that it matches (s? before |serv. Recall, you have to order by decreasing complexity.
.
In the end, the pattern better reads: /[\W\d]ad(serv(er|e)?|s|v)?[\W\d]/

NJH · Joined: 13 Nov 2003 Posts: 183 Location: Hampshire, England

rue,

Interesting comments. I will change my filters to [\W\d]. I was going to have a go sometime about re-working my filter to get the ? of the s? outside the OR set, but what I posted I tested following our last correspondence. I do not necessarily understand why. My filter matches /ad/, /ads/, /ad0 and any other number, /adv/, /adserv/, /adserve/ and /adserver/. It also matches other leading and trailing characters.

If you can explain why it works when I could not get something similar to work in our previous correspondence I would be interested.

Nick

rue · Developer Joined: 22 Oct 2003 Posts: 752

NJH:
In the JavaScript Console, try these two cases:

alert("vx".match(/(s?|v)/));
alert("vx".match(/(s?|v)x/));

The first correctly reports an empty, yet positive match: s? matches true for zero-occurrances on the first character it encounters, the set stops testing.
.
The second, however, catches the "v". Apparently sets continue testing if something follows the set. Whether this is a hidden-rule, or a bug, it's rather illogical. I wouldn't advise relying on it.

Guest · Posted: Wed Dec 17, 2003 Post subject:

Does anyone have a simple, but effective filter that WON'T kill http://www.pvponline.com ?

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest (tychoquad):
Scroll up, and copy the entries posted here.
.
Then, add these:

/[\W\d](ad|dime|double|fast|value|click)(stream|s|thrutraffic|thru|xchange|click)[\W\d]/
http://pagead2.googlesyndication.com

Guest · Posted: Wed Dec 17, 2003 Post subject:

Thank you rue, this is exactly what I was after.

The only simple/effective list i have been able to find that didn't kill all images in existance was that mac one you posted a while ago. all the others were so long, it lagged my whole computer every time firebird loaded a page.

A big problem here, is that I don't think anyone exept you knows how to properly build a short and effective list. I have no idea what all this expression stuff is about, even after reading the stuff at adblock.mozdev.org

Henrik · Posted: Wed Dec 17, 2003 Post subject:

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest:
It just takes a little patience -- thinking things through.
.
Having access as we do "behind-the-scenes", Henrik and I are acutely aware of the overhead for larger lists. For elements that aren't blocked, the entire list must be tested against them; if you're not careful, the penalty can be significant.
.
Check out the RegExp tutorials we came up with for Bethrezen, here. You'll find many links to further tutorials, as well.

TomChiverton · Joined: 11 Nov 2003 Posts: 3

Please don't but 'banner.*' in the generic block list - perfectly nice sites often call or serve their nav. banners like that :-)

Guest · Posted: Tue Jan 06, 2004 Post subject:

Guest · Posted: Wed Jan 07, 2004 Post subject:

also how would i set up my filters so that they do not block http://winamp.com/images/home/winamp5ad.jpg

(i am using rue's filters posted earlier in this thread.)

thx

kstahl · Posted: Thu Jan 08, 2004 Post subject:

Change
/[\W\d]ad(server|s)?[\W\d]/
to
/\Wad(server|s)?[\W\d]/

( removing the first \d )

Guest · Posted: Fri Jan 09, 2004 Post subject:

Guest · Posted: Sat Jan 10, 2004 Post subject:

Hi all,

I've met another false positive problem with this same rule, but you need to control the server side to reproduce it at will. Let's say :
my_site.com/something.php?PHPSESSID=70b25adf7aeb5

The "ad" inside the hexa parameter will be filtered by the rule.

I have fixed this using :
/[\W\d]ad(server|s)?[\W\d](\/|\.)/
That is, specifying that this must be followed by a slash ( a folder) or by a dot (a file).
Now my problem is that by reading this thread, it seems that passing "ad=..." as a parameter is sometimes used. Or is it ?

Your comments ?

Another problem is that neither versions of the rule filter content from www.smartadserver.com, which I don't really understand.

kstahl · Posted: Sat Jan 10, 2004 Post subject:

/[\W\d]ad(server|s)?[\W\d](\/|\.)/

The [\W\d] part _requires_ a non-alphabet character or a digit after the adserver part of the name _followed_ by a slash or a period.

www.smartadserver.com will not be a match, but say www.smartadserver7.com would.

A modified version which would filter both is
/[\W\d]ad(server|s)?[\W\d]?(\/|\.)/

kstahl · Posted: Sat Jan 10, 2004 Post subject:

Sorry, brain fart.

A better variant would be:
/[\/\.]ad(server|s)?\d*[\/\.]/

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest:
/[\W\d]ad(server|s)?[\W\d]/ doesn't match your proposed url: ..=70b25adf7..
.
It would match if the url contained another digit immediately after "ad". To avoid this, just continue this thread's earlier edit, removing the trailing digit-match:
/\Wad(server|s)?\W/

NJH · Joined: 13 Nov 2003 Posts: 183 Location: Hampshire, England

kstahl · Posted: Sun Jan 11, 2004 Post subject:

rue · Developer Joined: 22 Oct 2003 Posts: 752

Earlier Guest:
You asked for my complete filter-list. I don't think it's necessary to post them all, but here's a few more, including some refinements of earlier postings:

http://pagead2.googlesyndication.com/ -- universally heinous -!
http://forums.mozillazine.org/images/avatars/5085478903f36e1443c9ee.jpg -- person i never want to see
yimg.com/*.js -- yahoo's ad-scripts..
us.yimg.com/a/ -- ..and ad-images
/\/buy_assets\//
/[\W\d](top|bottom|left|right)?banner(s|id=|\d)[\W\d]/ -- general banner-stuff (improved)

The improved breakup of NJH's super-filter (less unnecessary recursion)

/[\W\d](double|fast)click[\W\d]/
/[\W\d]click(stream|thrutraffic|thru|xchange)[\W\d]/
/[\W\d]value(stream|xchange|click)[\W\d]/
/[\W\d]dime(xchange|click)[\W\d]/

MW · Guest

MW · Guest

and this one too....

/\/pagead2.googlesyndication.com\//

rue · Developer Joined: 22 Oct 2003 Posts: 752

MW:
I know you wouldn't inherently know this, but simple-filters which include a complete host-name are matched faster than any other filter. This is because all such filters' hostnames are copied into a hash and key-matched against the element's host.
.
So http://pagead2.googlesyndication.com/ matches faster than /\/pagead2.googlesyndication.com\//.

MW · Guest

thanx for the tip on the filter http://pagead2.googlesyndication.com ... i didn't realize that......

BTW what'a up with http://forums.mozillazine.org/images/avatars/5085478903f36e1443c9ee.jpg
she break your heart ??? Sad

ValkRaider · Guest

Has anyone looked at the Privoxy default.actions file and tried to incoporate some of their rules into Adblock rules?

It seems like they could be pretty good, I had good luck with Privoxy on the computers I have used it on... (I like Adblock a bit better as a general style issue though).

Here are privoxy's "general" rules:

rue · Developer Joined: 22 Oct 2003 Posts: 752

Look closely at that list -- in particular, here:

Guest · Posted: Mon Jan 19, 2004 Post subject:

Markfive · Guest

Here's an Amazon filter I created.. any ideas/sugesstions?

/amazon\.com.*\W(promotions|marketing|merchants|stores|associates)\W/

Dan · Guest

er, ok, having read the above, I'm none the wiser...

what would be suggested as the best basic adblock filter list then?

rue · Developer Joined: 22 Oct 2003 Posts: 752

Dan:
I'd say this:

rue wrote:

/\D\d{2,3}x\d{2,3}\D/ -- catches banner-sizes in source: 192x74

..and these.

Guest · Posted: Sun Jan 25, 2004 Post subject:

NJH · Joined: 13 Nov 2003 Posts: 183 Location: Hampshire, England

Going on the observation that most or all ads I've seen do not start with a number, I no longer use /[\W\d] at the start of my filters. I normally use /\W. This rule is the one exception where I use /[\W_] at the beginning because I have seen ads of the form "something_ad/somethingelse" and this still picks them up. This should allow your URL.

HTH

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest:
Regardless of prefix-digits' commonality, winamp5ad is a very unique exception. If a legitimate picture was named adimage.jpg, it too would be filtered. And I'd offer no apology.
.
If the extreme-exceptions seem bothersome, Adblock likely isn't your cup of tea.

Guest · Posted: Mon Jan 26, 2004 Post subject:

no its ok i was just wondering if there was an easy way to fix it. guess not. well then maybe a whitelist could be developed?

NJH · Joined: 13 Nov 2003 Posts: 183 Location: Hampshire, England

Guest, you wrote

rue · Developer Joined: 22 Oct 2003 Posts: 752

NJH:
It's not that I dislike your variant. Rather, I just think it unnecesary.
.
I've seen a few false-positives with my filters. But the urls being caught in each case were absurdly close to what an ad might be. Rather than cripple the effectiveness toward catching real ads, I'm happy to entertain a few wrong matches.

.
ps: yes, whitelists are coming. hang in there.

Guest · Posted: Tue Jan 27, 2004 Post subject:

sorry. I missed your post. Thanks for the suggestions. I'll try that.

jim · Guest

/.{150,}/

only webbugs need that many characters.

Guest · Posted: Wed Feb 11, 2004 Post subject:

taken from MyIE2. i haven't seen a single ad with this list:

Guest · Posted: Wed Feb 11, 2004 Post subject:

maybe I'm not understanding how adblock works but wouldn't that list slow down page load time?

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest:
Adblock automatically ignores head / end wildcards for Simple Filters.
.
What's more, using them (as above) provides an easy way to include the forward-slash (/) at both ends of the filter -- without making a regexp.

Melgund · Guest

Is a longish list of simple expressions slower than a much shorter set of regular expressions that block the same stuff? Question

rue · Developer Joined: 22 Oct 2003 Posts: 752

Melgund:
Regexp can compress "patterns" for speed. But, for unique terms, it wouldn't do much.

Ex: ad, adstuff, adworld -> slower than -> /ad(stuff|world)/
:
Ex: un, related, terms -> nearly same -> /un|related|terms/

Guest · Posted: Fri Feb 13, 2004 Post subject:

Guest · Posted: Fri Feb 13, 2004 Post subject:

heres my list

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest · Posted: Sat Feb 14, 2004 Post subject:

Hi. I use Agnitum Outpost firewall & can see filtered adverts in the log. Here's the statistics for 2 days' usage (~ 2.5 hours):

rue · Developer Joined: 22 Oct 2003 Posts: 752

Guest:
Excellent post. I'd never seen anything but pagead2.google.. -- the filter's now generalized to the principal domain.
.
Adblock filters with the JavaScript test() method, which means backrefs are basically ignored. Blocking them just adds to the character-load overhead -- pref-storage / retrieval.

Tristan2 · Joined: 08 Jan 2004 Posts: 1 Location: Germany, Osnabrueck

rue · Developer Joined: 22 Oct 2003 Posts: 752

Tristan:
To clarify: "Advanced filters" is just a label chosen for the extracted list -- to set it apart.
.
Per your filter: "advertisement" has a trailing underscore. Just copy what you did at the beginning.

/[\W\d_]ad(server|s|vertisement|vertising|vert|river)?[\W_]/

Guest · Posted: Sun Feb 22, 2004 Post subject:

I have done some snooping through the page source for Geocity sites and I've come up with the following filter to get rid of the Geocities ads: .geocities.com/js_source

I don't understand why it wasn't added before but there is probably a reason and I'm going to look like an idiot

MW · Guest

that might interfere with some of the page functionality... the filter i use for geocities is

Guest · Guest

It seems this filter:

Guest · Posted: Wed Mar 03, 2004 Post subject:

I use Org's filters plus a acouple site specific ones. Works great.

Guest · Posted: Thu Mar 04, 2004 Post subject:

just FYI google has changed its ad scheme. Not sure how its done now... I HATE THOSE ADS!

adave · Guest

kstahl · Posted: Fri Mar 26, 2004 Post subject:

You could use [\W\d_] instead of [^a-zA-Z], it looks much cleaner. Also, you don't need [Aa], Adblock isn't case-sensitive.

asd · Guest

Org · Joined: 23 Oct 2003 Posts: 349

The string "banner_id" inside "<a href>" doesn't match, because your regexp doesn't match the "_".

The string "banner" inside "<img>" doesn't match, because your regexp requires it to be followed by one of the strings "s", "it", "forum", "id=" or a single digit.

Try this one:

/[\W\d](top|bottom|left|right)?banner(s|it|forum|_?id=)?[\W\d]/

asd · Guest

OK, I have now made some general block rules. Please tell me if someting is not correct:

/\W(top|right|bottom|left|side)?ad(forum|click|s|server)\W/

So that ?-symbol makes it optional to ad-word to have those top,right-words in front? But is that \W also optional. So now it would also things like sadforum? And how to make that also possible to block ad-words only like /ad/? Should I put \W in those brackets too?

Guest · Posted: Wed Mar 31, 2004 Post subject:

kstahl · Posted: Wed Mar 31, 2004 Post subject:

(Yes, it was me that posted the above.)

Here's a decent page on regexps:
http://www.regular-expressions.info/reference.html

These are some of my own filters:

hri · Guest

Works great for me:
My strategy is to get as many ads as possible through generic extensions on ad, banner, promo & click. For the rest I rely on domain or directory specific filters.

[Adblock]
/(new|double|fast|value)click./
/[\/.]ban(ner|nerfarm|neri|image|source)s?[\/.]/
/[\/._-](|dhtm|i)ad(banner||mentor|s|s2|sdk|sv3|server|trix|image|img|log|vt|bureau|counter|v|vert|vertising|vertisement)s?[\/._?-]/
/[\/._]promo(|tion)s?[\/._]/
/[\/](associates|affiliates|us.yimg.com\/a)[\/]/
/[\W\d\/.](203.199.70.2|2o7|atdmt|atwola|bfast|bluestreak|coremetrics|dgm2|falkag|hitbox|marketbanker|qksrv|ru4|tribalfusion|zedo)[\W\d\/.]/

asd · Guest

Thank you kstahl. You answered my questions perfectly.

kstahl · Posted: Wed Mar 31, 2004 Post subject: Re: Try this

asd: You're welcome. I just noticed I messed up though. My example /adbanner6.gif would of course not get blocked, as the "banner" part is missing in the filter. D'oh!

Org · Joined: 23 Oct 2003 Posts: 349

kstahl · Posted: Thu Apr 01, 2004 Post subject:

Thanks, I must've missed that. I guess it makes sense when you think about it.

hri · Guest

Thanks Org.

kStahl, I like your "/\D(588|468|234|120)x600?\D/ ". Thanks.

I might add sizes to my list to handle the ads I havent been able to get rid of.

Guest · Posted: Thu Apr 01, 2004 Post subject: Problem with filter

I noticed the filter ( /\D\d{2,3}x\d{2,3}\D/ ) tends to vlock a lot of site logos that are not ads at all. In most, if not all cases, these logo's actually have the word "logo" in the file name. Could someone purpose a filter that would work the same but exlude images with the word logo in them?

kstahl · Posted: Thu Apr 01, 2004 Post subject: Re: Problem with filter

Guest · Posted: Thu Apr 01, 2004 Post subject: Re: Problem with filter

Guest · Posted: Thu Apr 01, 2004 Post subject:

I've extended your filter a little to include the examples from the firewall log above. It seems to be working nicely.
Where did you get your size related numbers from, just personal experience?

kstahl · Posted: Thu Apr 01, 2004 Post subject:

Yes, the sizes are just from personal experience and the web pages I regularely visits.

The ? is making the second 0 in 600 optional so it matches 60 too.

Guest · Posted: Thu Apr 01, 2004 Post subject:

What do you think of /\D([^l][^o][^g][^o]).*\d{2,3}x\d{2,3}\D/
I THINK this will block anything in the form (number)x(number) unless it has the word logo in it? Let me know if this will work as I expect it.

Guest · Posted: Thu Apr 01, 2004 Post subject:

Actually, that doesn't work at all. Isn't there any way to negate something so that the filter will fail if it is in it? like !(logo) or something? I'm looking at the regular expression definitions at http://www.regular-expressions.info/reference.html and I don't see anything. It doesn't make a lot of sense to not have a not in regular expressions.

Guest · Posted: Thu Apr 01, 2004 Post subject:

Ok, I think I have it now:
/\D(?<!logo).*\d{2,3}x\d{2,3}.*(?!logo)\D/
I THINK this will block anything in the form numberxnumber (only 2 to 3 digit numbers) unless it contain a logo befor or after it. It seems to work. Could anyone find an ad in the form numberxnumber for me so I can test it out?

kstahl · Posted: Fri Apr 02, 2004 Post subject:

http://www.aintitcoolnews.com has several flash banners of that type on the front page.

Guest · Posted: Fri Apr 02, 2004 Post subject:

Ok, my reg exp doesn't seem to be working right. It's not blocking those flash ads. It's advanced, so I wouldn't be suprised if Adblock doesn't parse it as I expect it to. I've broken down and used a more specific filter:
/\D(((588|468|234|120)x600?)|88x31|100x100|125x125)\D/

Erigion · Guest

Ok I imported the guest's adblock filter list, who used his firewall to determine what to block.

NJH · Joined: 13 Nov 2003 Posts: 183 Location: Hampshire, England