I'm working on a tool to generate SPF TXT records and I am trying to get a loop for scanning the domain for the relevant hosts A records. All I get is just a single entry when I supply the domain name as the domain to check. The other hosts in the domain don't return entries...
Any ideas on how to do this?
Did some research and it seems that you need to do a zone transfer to get the records, as allowing a random IP to access the whole domain has too many security vectors. So no real way to do what I was wanting... Sorry for wasting time!
Related
Like many others, I have navigated the SPF/DKIM/DMARC world with some confusion.
About 4 weeks ago or so I finished setting everything (SPF/DKIM/DMARC) up correctly for a GoDaddy-hosted domain that uses Google's mailservers.
I set the _dmarc TXT record to take zero action with p=none and I used Postmark to monitor the results to see what was passing and failing over a week.
After a week or so I looked at the Postmark results and inserted the include: statements for the domains that I wanted to pass, but weren't. Then I waited another week to see the results. However, the results showed that the domains still weren't passing SPF or DKIM. Below is the SPF record, I've redacted parts of it that are revealing, but two of the domains are legit and still aren't passing.
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
Do I need to use the actual IP addresses in the include statements instead of the domains? Postmark lists these as well so that would be easy if so.
No, you shouldn't copy their IPs in there because they are subject to change, especially Google's.
If it's failing, presumably you have some results (usually in message headers) that tell you exactly which IP is failing, and you can track it down manually though those includes, do a reverse lookup on it, etc.
However, you're also using GoDaddy, which is mostly guaranteed not to work as they either block outbound SMTP or route it through their own servers, so you're very unlikely to get an SPF pass.
The issue was with SPF DNS lookup limits. I had no idea this was a thing and I'm amazed that this isn't mentioned anywhere on the documentation (whether that's Google's official documentation or otherwise) on setting up SPF/DKIM/DMARC, and didn't come up in Googling of this issue. This limit is designed to prevent denial of service attacks and infinite DNS loops.
For anyone else who sees this post
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
This SPF record actually has almost 15 DNS lookups, and the limit is 10 per domain. You can find out how many SPF DNS lookups your domain has with a service like AutoSPF or Easy DMARC
The solution, once you see your total DNS lookups, comes in four options:
Create subdomains and use those to diversify the records. For example using "email#business.mydomain.com" as the email for freshemail.io. Then on the SPF record for that subdomain, you would only have v=spf1 include:freshemail.io resulting in less than 10 DNS lookups for that domain.
As #Synchro mentioned, you don't want to use IPs because those can very well change, but the concept of using IPs instead of the domain names does essentially work because an IP address doesn't cost a DNS lookup. Check with the support/engineering of whatever service you're using, it's possible that they have an IP (or an IP range) that doesn't change often. You might be able to bring your DNS lookups under ten using this.
Note that Google takes up about 3 DNS Lookups, and you'll probably want to leave that one as the _spf.google.com value
Note that every SPF record also has a 255 character limit, so if you're using only IPs you'll need to break that up into a lot of SPF records probably
Use an SPF flattening or compressing service like AutoSPF. Essentially, these services employ method #2, but do some backend work every few hours to check and update the IP addresses associated with the domains. Then they provide you with a "compressed" record like v=spf1 include:_6359384.autospf.com ~all that references all of your records and results in far fewer DNS lookups.
Create your own method that acts kind of like #2 and #3, using GoDaddy's API and brew up something that performs updated lookups on a schedule/job and updates separate SPF records including all of the IPs.
I incidentally created a wrong domain, or after some testing I wanted to delete the domain.
Should I do that? and How?
It's strongly recommended not to delete domain.
There could be some data like tasks associated with a domain in the Cadence system. There is no tooling to clean them yet. Simply delete the domain will lead to corruption. For example, there may be a timer task scheduled for 1 year later in that domain. If the domain is deleted, it may look okay right now. But one year later, when the timer fire, the system will be corrupted. By design Cadence needs to be strongly consistent. So the server cannot simply skip a timer task.
In most cases, you don't need to delete an existing domain. As long as you don't use it you are fine. Including cases like you created a domain with a wrong name, or you want to deprecate a domain. In those cases, just don't bother it.
Another case, you created a local domain but later on realized that it should be a global domain. It's recommended to just ignore the local domain.
There could some slightly better reasons to delete a domain. For example, in the above case, you want to keep using the same domain name for whatever reason.
!!Danger Zone!!
The ONLY case that you can delete a domain is the case that you are sure the domain has never been used at all.
The operation is as follow. Using your database tool:
For SQL:
DELETE FROM domains WHERE name ="<yourDomain>" LIMIT 1
For Cassandra:
SELECT domain FROM domains_by_name_v2 WHERE domains_partition=0 AND name ="<yourDomain>"
This will return the domainUUID.
Then delete the records from two table:
DELETE FROM domains_by_name_v2 WHERE domains_partition=1 AND name ="<yourDomain>" LIMIT 1;
DELETE FROM domains WHERE id = domainUUID LIMIT 1;
Let's say I have 10,000 usernames in the RDBMS database which will use the XMPP chat service like Ejabberd or MongooseIM that I will put manually in the database. Now every 1000 usernames in the above 10,000 have fixed prefix in their JID usernames like:
1-1000 usernames start with aa__ (Example - aa10001, aa10002 , ...)
1001-2000 usernames start with bb__ (Example - bb10001, bb10002, ...)
2000-3000 usernames start with cc__ (Example - cc10001, cc10002, ...)
and so on ... total of 10 different prefix usernames
Now I just want that these 10 different pairs must not know others prefix usernames pairs and must not be able to send any type of stanza (like message, presence, IQ) to other pair (having a different prefix)
Possibilities:
If I can use some regex in the block policy then that would be great instead of blocking the individual.
I know u may suggest of using a different Virtual host for each pair. But managing so many Virtual hosts can be difficult because in the above example 10K is just a small example of a bigger picture.
Please let me know if question is not clear to you?
I just want that these 10 different pairs must not know others prefix usernames pairs and must not be able to send any type of stanza (like message, presence, IQ) to other pair (having a different prefix)
First of all, play with mod_filter. Looking at its example configurations, probably you can accomplish what you want using it:
https://github.com/processone/ejabberd-contrib/tree/master/mod_filter
Maybe you can define ACL based in the usernames patterns ("aa*" or something like that), and this way you can have all users in a single vhost.
In case the only solution requires having a vhost for each user group, some ideas:
Now every 1000 usernames in the above 10,000 have fixed prefix in their JID usernames like:
As you mentioned, you can define a vhost in ejabberd for each user group: 10001#aa.example.com... Or in order to not change existing usernames: aa10001#aa.example.com...
Maybe if I can use Virtual host in some way that it is easy to manage (adding new pairs frequently) without adding a new database for each then that will be a best-ever solution
Right, you can have all the vhosts in a single database, if you configure ejabberd like this and use the apropiate *.sql file when creating the database:
new_sql_schema: true
Can I add a virtual host dynamically in the config without stopping the server?
Yes, it should work (it's a recent feature). Edit ejabberd.yml as you wish, and reload it with:
$ ejabberdctl reload_config
Where I can get a list of IP from spam blacklist database. I need something like phishtank API where I can download their blacklist and implement it with my app.
I have seen a spam website doing lookup like http://www.mxtoolbox.com/ I was hoping I could get access to spam feed like them. csv or sql would be nice.
Thank you.
That site is using DNSBL lookups, not a local resource. To perform such lookups, individual blacklists sometimes differ, but in general, to look up an IP address, reverse the octets, add the DNSBL zone, and resolve; a result indicates a match (typically 127.0.0.x where the x might be a result code of some sort). So for example, to look up 123.45.67.89 in dnsbl.example.net, you'd perform an A query like this:
nslookup 89.67.45.123.dnsbl.example.net
(The program nslookup is not particularly good or convenient, but it is widely available, even on Windows.)
Read up on DNS in general and DNSBLs in particular. I have found Wikipedia to have good overviews for these topics.
I have my own DNS server ex: IP 34.34.34.34.
This server knows about few domain names.
At times I need to send a TXT Record query to this server and process the result.
What is the best framework to use for this on iOS?
It should be fairly simple to do it in C and call it from ObjectiveC.
Here's an example I to get you started: https://gist.github.com/wil/6141275
Just make sure you link with -lresolv (i.e. libresolv.dylib)