Where I can get a list of IP from spam blacklist database. I need something like phishtank API where I can download their blacklist and implement it with my app.
I have seen a spam website doing lookup like http://www.mxtoolbox.com/ I was hoping I could get access to spam feed like them. csv or sql would be nice.
Thank you.
That site is using DNSBL lookups, not a local resource. To perform such lookups, individual blacklists sometimes differ, but in general, to look up an IP address, reverse the octets, add the DNSBL zone, and resolve; a result indicates a match (typically 127.0.0.x where the x might be a result code of some sort). So for example, to look up 123.45.67.89 in dnsbl.example.net, you'd perform an A query like this:
nslookup 89.67.45.123.dnsbl.example.net
(The program nslookup is not particularly good or convenient, but it is widely available, even on Windows.)
Read up on DNS in general and DNSBLs in particular. I have found Wikipedia to have good overviews for these topics.
Related
I'm working on a tool to generate SPF TXT records and I am trying to get a loop for scanning the domain for the relevant hosts A records. All I get is just a single entry when I supply the domain name as the domain to check. The other hosts in the domain don't return entries...
Any ideas on how to do this?
Did some research and it seems that you need to do a zone transfer to get the records, as allowing a random IP to access the whole domain has too many security vectors. So no real way to do what I was wanting... Sorry for wasting time!
Like many others, I have navigated the SPF/DKIM/DMARC world with some confusion.
About 4 weeks ago or so I finished setting everything (SPF/DKIM/DMARC) up correctly for a GoDaddy-hosted domain that uses Google's mailservers.
I set the _dmarc TXT record to take zero action with p=none and I used Postmark to monitor the results to see what was passing and failing over a week.
After a week or so I looked at the Postmark results and inserted the include: statements for the domains that I wanted to pass, but weren't. Then I waited another week to see the results. However, the results showed that the domains still weren't passing SPF or DKIM. Below is the SPF record, I've redacted parts of it that are revealing, but two of the domains are legit and still aren't passing.
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
Do I need to use the actual IP addresses in the include statements instead of the domains? Postmark lists these as well so that would be easy if so.
No, you shouldn't copy their IPs in there because they are subject to change, especially Google's.
If it's failing, presumably you have some results (usually in message headers) that tell you exactly which IP is failing, and you can track it down manually though those includes, do a reverse lookup on it, etc.
However, you're also using GoDaddy, which is mostly guaranteed not to work as they either block outbound SMTP or route it through their own servers, so you're very unlikely to get an SPF pass.
The issue was with SPF DNS lookup limits. I had no idea this was a thing and I'm amazed that this isn't mentioned anywhere on the documentation (whether that's Google's official documentation or otherwise) on setting up SPF/DKIM/DMARC, and didn't come up in Googling of this issue. This limit is designed to prevent denial of service attacks and infinite DNS loops.
For anyone else who sees this post
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
This SPF record actually has almost 15 DNS lookups, and the limit is 10 per domain. You can find out how many SPF DNS lookups your domain has with a service like AutoSPF or Easy DMARC
The solution, once you see your total DNS lookups, comes in four options:
Create subdomains and use those to diversify the records. For example using "email#business.mydomain.com" as the email for freshemail.io. Then on the SPF record for that subdomain, you would only have v=spf1 include:freshemail.io resulting in less than 10 DNS lookups for that domain.
As #Synchro mentioned, you don't want to use IPs because those can very well change, but the concept of using IPs instead of the domain names does essentially work because an IP address doesn't cost a DNS lookup. Check with the support/engineering of whatever service you're using, it's possible that they have an IP (or an IP range) that doesn't change often. You might be able to bring your DNS lookups under ten using this.
Note that Google takes up about 3 DNS Lookups, and you'll probably want to leave that one as the _spf.google.com value
Note that every SPF record also has a 255 character limit, so if you're using only IPs you'll need to break that up into a lot of SPF records probably
Use an SPF flattening or compressing service like AutoSPF. Essentially, these services employ method #2, but do some backend work every few hours to check and update the IP addresses associated with the domains. Then they provide you with a "compressed" record like v=spf1 include:_6359384.autospf.com ~all that references all of your records and results in far fewer DNS lookups.
Create your own method that acts kind of like #2 and #3, using GoDaddy's API and brew up something that performs updated lookups on a schedule/job and updates separate SPF records including all of the IPs.
I'm interested in how https://bgpview.io/asn/19551 got this information:
TRAFFIC RATIO: Mostly Outbound
I'd like to get more information about an ASN than is available on bgpview.io.
I didn't find any external source which allows me to get this information, what could I try?
This kind of information is not written anywhere. This is the result of a good guess based on multiple parameters, like the company name, the service they offer, etc. For instance, an ASN assigned to Cloudflare will have a traffic direction Mostly Outbound and a traffic speed of 100GB+.
There is obviously a lot of work behind this information, which is almost completely manually added.
BGPView.io takes this information from peeringdb.com
I can confirm this since I have my own AS212000 and when I defined my Traffic Ratio on peeringdb.com it got updated on BGPView.io as well.
My DNS provider works perfectly for A records.
I am having great difficulty understanding the syntax of SPF records. I have no prior experience.
The DNS provider supports SPF records and it has two control boxes for information: 'Name' and 'SPF data'.
The A record which functions fine looks like this:
Name: potsandpins.info
IPV4 Address: 45.61.228.207
The SPF record which is giving me no joy looks like this:
Name: potsandpins.info
SPF Data: "v=spf1 a -all" (including the quotation marks)
My emails are received with a red flag in Gmail which says 'Gmail couldn't verify that potsandpins.info actually sent this message'.
Can anyone suggest anything as I've tried all sensible permutations?
You don't seem to currently have an SPF record for potsandpins.info maybe you deleted it because you ran into trouble. Anyway, think of the SPF as a whitelist of any IP addresses or hosts you've given permission to send email on your behalf.
The name would be either the root domain, sometimes designated by the #, or a hostname, foo, which you'd use if you were sending email out as example#foo.example.com.
The SPF data would be the version number (v=spf1), then mechanisms (e.g., a), and then the ip addresses or hosts you'd like to authorize, then the qualifier such as -ALL, which intends a hard fail. You may want to back off from that using ~ALL for now, which intends a softfail. I think it's better to be specific in SPF records as then they're easier to follow exactly what they're authorizing.
Here's an example SPF record. Let's say you wanted to authorize 192.0.2.10 and Google.
v=spf1 ip4:192.0.2.10 include:_spf.google.com ~all
Let's say you wanted to authorize a range of IP addresses and MailChimp:
v=spf1 ip4:192.0.2.0/24 include:servers.mcsv.net ~all
Here's a good article on common mistakes in SPF records.
Then it's important to validate your SPF record using a tool such as the SPF Survey. I like this tool because it gives more detailed, actionable error messages when there's a problem.
if you post the full headers of an example email and indicate any other services you use to send email, then it would be possible to provide more specific advice. For future reference, it's best to provide more details when you post to Stack Overflow as that makes it easier to help. I tried in this post but the information you provided limited how specific the answer could be.
Also, for future reference, it's best to post using example.com rather than a real domain name and use IP addresses from an IPv4 block reserved for documentation.
The blocks 192.0.2.0/24 (TEST-NET-1), 198.51.100.0/24 (TEST-NET-2),
and 203.0.113.0/24 (TEST-NET-3) are provided for use in documentation.
Anyway, I hope this helps.
I am making a resolver and a nameserver program with out using dns libraries(such as netdb.h) by directly sending a dns message. but i have few problems. As far as i find out when we send a recursive request, the nameserver queried finds out the Records for us. Does the queries used by nameserver to query other servers, are similar to iterative queries? at least these images suggest this.
I am also confused if a client can do a iterative query or only a nameserver can do iterative queries?
Recursive dns lookup:
Iterative dns lookup:
Any DNS client (or "resolver") may perform iterative queries.
By definition, though, a resolver that does perform iterative queries is a recursive resolver, and not a stub resolver.
Stub resolvers are usually implemented as libraries, linked directly into your executable.
However it is also possible to build a complete recursive resolver as a standalone library. libunbound is a particularly good example.
A client can certainly do iterative queries on its own without needing to consult a recursive resolver but there are many reasons not to do that:
simplify the complexity of the software that needs to exist in the stub resolver libraries (e.g. libresolv or built into libc) on every host
delegate iterative querying to a server in an ISP network or nearer to a "backbone" which will have a better Internet connection (most importantly, lower delay) and can complete the iterative query faster.
aggregate the DNS queries of many end users onto a small number of caching resolvers. Most of the time the resolvers won't have to do the full iterative query: they will have some or all of the results already cached.
reduce the number of places where the "hints" file (a list of root nameservers and their IP addresses), which is necessary to bootstrap a recursive resolver, has to be deployed.
DNSSEC throws a wrench in that: with DNSSEC, the end user must perform the full iterative query if it wants to certify the result. It is yet to be seen how the large scale deployment of DNSSEC-enabled resolvers will happen.
a recursive query:- the DNS server might send query other DNS server's in the internet on your behalf, for the answer.As proxy server sends the query to the main server for the answer.
In an iterative query, the name server, will not go and fetch the complete answer for your query, but will give back a referral to other DNS server's, which might have the answer.
it the proxy server gives you the answer unless it gives to search in other servers