Add other domains owned by admiral?

[KeenRivals]

Many other domains were found that are owned by Admiral and point to the same IP as #1. There's a list at https://pgl.yoyo.org/adservers/admiral-domains.txt

[paulgb]

Thanks for this! I'll have a look when I'm at a computer

…

On Aug 12, 2017 11:55 AM, "KeenRivals" ***@***.***> wrote: Many other domains were found that are owned by Admiral and point to the same IP as #1 <#1>. There's a list at https://pgl.yoyo.org/adservers/admiral-domains.txt — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#4>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAC0XRsMHcj3UPhAvHkJToJddeujsbYlks5sXcr6gaJpZM4O1cvW> .

[tofof]

That list is woefully incomplete. See my comment here for some analysis. Click the reverse-domain links I provide (e.g. ipv4info for functionalclam.com) and you can see how deep this rabbit hole goes, and that's just for a single ip.

Sample admiral domains not in that list:
btez8.xyz
innocentwax.com
completecabbage.com
4jnzhl0d0.com
h78xb.pw

It's trivial to observe hundreds of Admiral domains, they probably number in the thousands.

[mvasilkov]

Let's kill the whole Admiral thing with fire!

[paulgb]

I've merged #8 which adds more Admiral domains, if there are still some missing I'm happy to add them as they are discovered

[melv-n]

I support adding all domains pointing to the same content - it's likely all these domains would eventually have found their way into EasyList in the first place. Might be less likely now.

[JamyDev]

Maybe block that IP too in anticipation of more domains being added?

[tofof]

#8 is still not even close.

It has...

• Some but not all - e.g. breezybath.com - of the domains currently on 104.155.48.223 (the same ip that unknowntray.com is served from)
• Some but not all - e.g. axiomaticalley.com - of the domains currently at 146.148.6.205
• Some but not all - e.g. btez8.xyz - of domains currently at 35.186.249.84

It's missing...

• owlsr.us and 65+ other domains that have been or are currently on 104.196.29.119
• roastedvoice.com and 64 friends from 146.148.79.27
• 17ars.xyz and past domains hosted by 52.39.21.213 but not photos.hollywood.com that's there now
• lewdwind.com and 20+ domains formerly at 104.155.177.112
• spottysense.com and many formerly at 54.187.68.112
• and more
• and more
• and 30 more
• and commoncannon.com and a dozen more
• and more
• and chickenstation.com and 100 others
• and these
• and those
• and frailoffer.com and friends
• and granodiorite.com and others
• and 43 more except there's an innocent one too

... I think that demonstrates my point. (Yes, there are many duplicates once you start putting all of these together).

If someone actually wants to make a serious attempt, which hasn't happened yet, just walk the related domains on a tool (like threatcrowd) that lets you do so easily.

It's trivial to verify these even when you're learning the naming patterns, since they all serve up the same image. But you really have to do the verifications. Bannersnack.com, for example, is NOT an Admiral domain, even though it was hosted alongside them once.

Some starting points that I haven't already exhausted above include...
tzwaw.pw
0D7DK.XYZ
pz37t.xyz
3jsbf5.xyz - beware, there's at least one domain (apstylebook.com) that'd be a false positive.
4jnzhl0d0.com
82o9v830.com
familiarfloor.com

The biggest problem is that they use google/amazon hosting and so you can't trivially blacklist everything that resolves into their IP space, and that tools like ipv4info, threatcrowd, alienvault, tcpiputils, all have incomplete datasets. You really need multiple people using different toolsets walking the same space to root all of these out.

[lol768]

@tofof Given the ones that I have seen seem to use valid SSL certs from Let's Encrypt, do you think crawling the CT logs is a viable way of checking for these?

[anon182739]

What about doing it the other way around? As you said, it's trivial to verify a domain. What about checking all suspicious domains? If one is found, it's sent to some central server to be added to the list. Another way would be to just blacklist the IPs. This would have some false positives, but if it becomes a default filter for ublock/adblock it would have the effect of forcing them to clean up their IP ranges. But really, how often do you have a legitimate reason for loading random .js from a .xyz domain?

@lol768
They issue hundreds of thousands of certificates each day, but if you manage to get the first filtering down it's feasible, then you can use DNS to filter out more.

@mvasilkov
I don't think that's feasible. Maybe by sending abuse notices to the registrars, since they care more about quantity than quality.

[anon182739]

Also, playing cat-and-mouse with domains is not really a problem. It costs at least $1 to register one, and much less to find and block it.

[paulgb]

Actually, @LOL786's suggestion sounds like a great idea for keeping track of new registrations from this company. It could be automated as a nightly script.

[paulgb]

That's just a way of narrowing down the search space; we'd still verify that the domain behaved like the others.

I'd like to keep this to a pure domain blacklist, as opposed to running code on the client, for a few reasons: 1. it's more portable to existing blocking extensions, 2. it's more performant, and 3. I'm more comfortable with the legal defensibility (I am confident that a passive blacklist can never be illegal; I don't want to speculate about the legality of a more active client-side approach)

[anon182739]

@zymase
You can just check the image dimensions, or the length of the body. This works for me:
def isAdmiralDomain(domain): try: response = requests.get('https://' + domain) except: return False if len(response.text) == 179: return True else: return False

Otherwise, how on earth can a list be conceived otherwise than with try-and-succeed/try-and-fail policy? It's just impossible. We are facing networks as dirty as bots.

Start of with the LE cert list. Filter away anything that doesn't end with .com, .xyz, or .pw. Issue DNS queries for everything. If the whois isn't protected, remove it. If they're not using whatever registrars they're using, remove it. Then use tor/proxies (it's kind of counter productive for them to block IPs) for the final verification.

Why not literally treat them as bots? I don't have any, but SMTP accounts are allegedly quite cheap. What if you'd just send bulk abuse notices to the registrars and accuse them of being C2 servers for some botnet? They generally don't care if the notices are valid, they only care about the quantity of notices they're getting.

@paulgb

That's just a way of narrowing down the search space; we'd still verify that the domain behaved like the others.
And this job can be automated easily, as long as the domain can be identified once you can put it in the list forever.

[anon182739]

GitHub stripped away the formatting, so you need to add some newlines and indentation.
The images are identical right now.
CRC32: 8db019c1
MD5: 681e062bb33b9ba28f3427e7283c81a8
SHA1: 3fcf7e14e92043a00926d340d45778b618bc87a9
SHA2: 32afacb9285649aa4af43ea03e7cd9a522aa3e6d0554a2dabe308fac4531be5f

[anon182739]

Another identifying mark is the robots.txt, which is unlikely to change:
User-agent: * Disallow: /

The 404 page seems nonstandard:
404 page not found
(it's not HTML)
Content-Type:"text/plain; charset=utf-8"

The registrar is enom for the .com domains and namecheap for the .xyz and .pw domains.

They all use the same 4 nameservers. You should be able to enumerate from that, it's an uncommon combination.
NS-1212.AWSDNS-23.ORG
205.251.196.188
NS-1627.AWSDNS-11.CO.UK
205.251.198.91
NS-305.AWSDNS-38.COM
205.251.193.49
NS-697.AWSDNS-23.NET
205.251.194.185