When browsers look up hostnames, do they do that even when the "host name" is an IP address? For instance http://74.125.39.105/ goes to some Google servers. Will this result in a DNS lookup on common browsers and platforms, such as Safari, IE, Firefox, Chrome, Opera, Windows, Linux, Mac OSX?
Or will the browser (in the common case) just start a connection directly without trying a DNS lookup first?
There is no DNS lookup done if the entire host part is numeric in nature. It could be in dotted quad format, or a single unsigned 32 bit integer. I haven't tested IPv6 yet.
There's no point in performing a lookup as there are no authoritative DNS records for these numeric "names".
Some recursive nameservers (e.g. dnscache) will respond to these kind of queries with the corresponding IP address, as if there was a real record of the form:
74.125.39.105. IN A 74.125.39.105
but this is, I assume, intended to deal with simplistic clients that assume everything is a DNS name. The most common nameserver, BIND, doesn't have this behaviour though - so don't rely on it.
Related
Like many others, I have navigated the SPF/DKIM/DMARC world with some confusion.
About 4 weeks ago or so I finished setting everything (SPF/DKIM/DMARC) up correctly for a GoDaddy-hosted domain that uses Google's mailservers.
I set the _dmarc TXT record to take zero action with p=none and I used Postmark to monitor the results to see what was passing and failing over a week.
After a week or so I looked at the Postmark results and inserted the include: statements for the domains that I wanted to pass, but weren't. Then I waited another week to see the results. However, the results showed that the domains still weren't passing SPF or DKIM. Below is the SPF record, I've redacted parts of it that are revealing, but two of the domains are legit and still aren't passing.
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
Do I need to use the actual IP addresses in the include statements instead of the domains? Postmark lists these as well so that would be easy if so.
No, you shouldn't copy their IPs in there because they are subject to change, especially Google's.
If it's failing, presumably you have some results (usually in message headers) that tell you exactly which IP is failing, and you can track it down manually though those includes, do a reverse lookup on it, etc.
However, you're also using GoDaddy, which is mostly guaranteed not to work as they either block outbound SMTP or route it through their own servers, so you're very unlikely to get an SPF pass.
The issue was with SPF DNS lookup limits. I had no idea this was a thing and I'm amazed that this isn't mentioned anywhere on the documentation (whether that's Google's official documentation or otherwise) on setting up SPF/DKIM/DMARC, and didn't come up in Googling of this issue. This limit is designed to prevent denial of service attacks and infinite DNS loops.
For anyone else who sees this post
v=spf1 include:_spf.google.com include:freshemail.io include:cherryroad.com ~all
This SPF record actually has almost 15 DNS lookups, and the limit is 10 per domain. You can find out how many SPF DNS lookups your domain has with a service like AutoSPF or Easy DMARC
The solution, once you see your total DNS lookups, comes in four options:
Create subdomains and use those to diversify the records. For example using "email#business.mydomain.com" as the email for freshemail.io. Then on the SPF record for that subdomain, you would only have v=spf1 include:freshemail.io resulting in less than 10 DNS lookups for that domain.
As #Synchro mentioned, you don't want to use IPs because those can very well change, but the concept of using IPs instead of the domain names does essentially work because an IP address doesn't cost a DNS lookup. Check with the support/engineering of whatever service you're using, it's possible that they have an IP (or an IP range) that doesn't change often. You might be able to bring your DNS lookups under ten using this.
Note that Google takes up about 3 DNS Lookups, and you'll probably want to leave that one as the _spf.google.com value
Note that every SPF record also has a 255 character limit, so if you're using only IPs you'll need to break that up into a lot of SPF records probably
Use an SPF flattening or compressing service like AutoSPF. Essentially, these services employ method #2, but do some backend work every few hours to check and update the IP addresses associated with the domains. Then they provide you with a "compressed" record like v=spf1 include:_6359384.autospf.com ~all that references all of your records and results in far fewer DNS lookups.
Create your own method that acts kind of like #2 and #3, using GoDaddy's API and brew up something that performs updated lookups on a schedule/job and updates separate SPF records including all of the IPs.
Say I've got a file descriptor from socket(2) and I've done a connect(2) on it -- is there any way later to determine (from inside the running program) what network device might be in use for the underlying transport? A call to stat(2) on the fd gives a device number of 0; none of the ioctl(2) or getsockopt(2) options seem applicable.
There's no foolproof way to do so -- certainly not a posix-compliant way.
However, in practice, you can easily determine the interface 99% of the time. After you've done the connect, use getsockname to obtain the IP address, then look through the list of available interfaces on the box for one with a matching IP address.
From the accept call you should be able to get the remote client's ip address (seen here on Beej's). Assuming you do not have any asymmetric routes, you can look up the route to that address in your local routing table. The routing table should tell you what ethernet device is used to send packets to the remote client.
EDIT:
you could use the following command line tool to query your local routing table with the remote client's address:
ip route get <remote-client-ip-addr>
I have a PostgreSQL table with a column (of type Text) 'ip_port'. This column has values like:
167.123.24.25:59874
This is IP:Port
And I'm trying to write a query to list all the subnets from those IP and a count of IPs in each subnet. Is there an elegant way of doing this in Postgres?
Thanks in advance!
You cannot do this - in PostgreSQL or any other tool - if the only information you have is the IP address. You must also know:
The broadcast address for the network;
The network subnet mask or mask prefix length; or
that pre-CIDR class-based address allocation is being used
Consider that a given IP is actually a member of multiple networks of increasing size. For the purpose of local area Ethernet-based IP networking there's a broadcast domain defined by the subnet mask, but there are also a nested set of increasingly broad routing aggregation domains for that address, some of which are visible on the public Internet and some of which are private to an internal network. The line between "public" and "private" can be blurry; for example, within my ISP's network their large network blocks are split into smaller chunks that are not visible from the outside world but are visible to the ISP's customers.
For example, a non-exhaustive list of networks for 167.123.24.25 might be, according to some quick calculations with the sipcalc tool:
167.123.24.25/32 (the host its self)
167.123.24.25/28: (a common ISP allocation): usable 167.123.24.17 - 167.123.24.30, netmask 255.255.255.240
167.123.24.25/24: (a common LAN subnet size): usable 167.123.24.1 - 167.123.24.254, netmask 255.255.255.0
167.123.24.25/20: (a midlevel routing aggregation): usable 167.123.16.1 - 167.123.31.254, netmask 255.255.240.0
167.123.24.25/16: The top-level APNIC allocation of this address block according to whois): usable 167.123.0.1 - 167.123.255.254, netmask 255.255.0.0
If you traceroute that address, consider that any of those routers might (but also might not) be the point at which the address becomes part of a narrower, more specific subnet. See Classless inter-domain routing on Wikipedia.
All that is before you even consider NAT (blech) and internal WAN routing. 167.123.24.25 might be a router for a whole hidden network of NATed hosts. These hosts are invisible to you from the outside without extremely sophisticated passive mapping that only someone close to them along their routing path can do.
To learn more about the address's network you have to do some serious probing to find out what addresses might be treated as broadcast addresses. Of course, most routers will filter packets destined for broadcast addresses from outside the local routing domain, so you might not be able to tell the difference between "host doesn't exist", "is a broadcast address" and "host exists but filters the packet I'm using for probing". Such probing is slow, and it'll also tend to make the network owners grumpy at you. Since this network is owned by the US Department of Public Works, I'd recommend not making them grumpy.
Where I can get a list of IP from spam blacklist database. I need something like phishtank API where I can download their blacklist and implement it with my app.
I have seen a spam website doing lookup like http://www.mxtoolbox.com/ I was hoping I could get access to spam feed like them. csv or sql would be nice.
Thank you.
That site is using DNSBL lookups, not a local resource. To perform such lookups, individual blacklists sometimes differ, but in general, to look up an IP address, reverse the octets, add the DNSBL zone, and resolve; a result indicates a match (typically 127.0.0.x where the x might be a result code of some sort). So for example, to look up 123.45.67.89 in dnsbl.example.net, you'd perform an A query like this:
nslookup 89.67.45.123.dnsbl.example.net
(The program nslookup is not particularly good or convenient, but it is widely available, even on Windows.)
Read up on DNS in general and DNSBLs in particular. I have found Wikipedia to have good overviews for these topics.
Below is a sample invoking of getaddrinfo()
status = getaddrinfo("www.example.net","1234", &hints, &server_info);
After that, the server_info will point to a linked list with all kinds of address information.
I have the following questions:
Since I have clearly specified the host name and port number, the only address infos I can think of are IPv4 and IPv6 addresses. So, the length of the linked list should be 2. Is there any other kind of address info besides them?
Thanks.
The name could resolve to more than one IPv4 or IPv6 address, there is nothing to say that only one IPv4 address will be returned, for example (try it with "www.google.com" for example, you'll likely get more than one IPv4 address).
But in any case, I think the basic premise of your question is wrong. Even if there was no possibility to return more than one IPv4 and one IPv6 address today, the function is documented to return an arbitrarily-long linked list of addrinfo objects. Therefore, even if your code worked today in every situation, there is no guarantee that it would continue to work tomorrow. If the function is documented to return an arbitrarily-long linked list, then you need to be able to handle that.
You want to disconnect the physical configuration of machines with names in your mind. DNS merely maps a name to a set of addresses. A lot of hosts will only have one interface. Many hosts will have multiple (called multi-homing). DNS doesn't care about what the configuration of the machine or machines that the addresses it maps a name to is. As simple examples often a server will have interfaces on multiple networks with different addresses that will all map to the same name. Sometimes when collapsing services from different machines onto one different names will map to the same address. So don't assume any sort of 1:1 mapping between names and machines much less interfaces.