聊聊HttpClient的DnsResolver

序

本文主要研究一下HttpClient的DnsResolver

DnsResolver

org/apache/http/conn/DnsResolver.java

/**
 * Users may implement this interface to override the normal DNS lookup offered
 * by the OS.
 *
 * @since 4.2
 */
public interface DnsResolver {

    /**
     * Returns the IP address for the specified host name, or null if the given
     * host is not recognized or the associated IP address cannot be used to
     * build an InetAddress instance.
     *
     * @see InetAddress
     *
     * @param host
     *            The host name to be resolved by this resolver.
     * @return The IP address associated to the given host name, or null if the
     *         host name is not known by the implementation class.
     */
    InetAddress[] resolve(String host) throws UnknownHostException;

}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

DnsResolver定义了resolve方法，可用于替换OS提供的DNS lookup

InMemoryDnsResolver

org/apache/http/impl/conn/InMemoryDnsResolver.java

/**
 * In-memory {@link DnsResolver} implementation.
 *
 * @since 4.2
 */
public class InMemoryDnsResolver implements DnsResolver {

    /** Logger associated to this class. */
    private final Log log = LogFactory.getLog(InMemoryDnsResolver.class);

    /**
     * In-memory collection that will hold the associations between a host name
     * and an array of InetAddress instances.
     */
    private final Map dnsMap;

    /**
     * Builds a DNS resolver that will resolve the host names against a
     * collection held in-memory.
     */
    public InMemoryDnsResolver() {
        dnsMap = new ConcurrentHashMap();
    }

    /**
     * Associates the given array of IP addresses to the given host in this DNS overrider.
     * The IP addresses are assumed to be already resolved.
     *
     * @param host
     *            The host name to be associated with the given IP.
     * @param ips
     *            array of IP addresses to be resolved by this DNS overrider to the given
     *            host name.
     */
    public void add(final String host, final InetAddress... ips) {
        Args.notNull(host, "Host name");
        Args.notNull(ips, "Array of IP addresses");
        dnsMap.put(host, ips);
    }

    /**
     * {@inheritDoc}
     */
    @Override
    public InetAddress[] resolve(final String host) throws UnknownHostException {
        final InetAddress[] resolvedAddresses = dnsMap.get(host);
        if (log.isInfoEnabled()) {
            log.info("Resolving " + host + " to " + Arrays.deepToString(resolvedAddresses));
        }
        if(resolvedAddresses == null){
            throw new UnknownHostException(host + " cannot be resolved");
        }
        return resolvedAddresses;
    }

}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

InMemoryDnsResolver实现了DnsResolver接口，它用一个ConcurrentHashMap来存放dns信息，提供add方法往map添加host及对应的ip地址，然后其resolve就是从这个map来读取对应的ip地址信息

SystemDefaultDnsResolver

org/apache/http/impl/conn/SystemDefaultDnsResolver.java

/**
 * DNS resolver that uses the default OS implementation for resolving host names.
 *
 * @since 4.2
 */
public class SystemDefaultDnsResolver implements DnsResolver {

    public static final SystemDefaultDnsResolver INSTANCE = new SystemDefaultDnsResolver();

    @Override
    public InetAddress[] resolve(final String host) throws UnknownHostException {
        return InetAddress.getAllByName(host);
    }

}
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15

SystemDefaultDnsResolver实现了DnsResolver，它用的就是jdk提供的InetAddress.getAllByName，默认是走的OS的DNS，可以通过sun.net.spi.nameservice.provider.去自定义

DefaultHttpClientConnectionOperator

org/apache/http/impl/conn/DefaultHttpClientConnectionOperator.java

@Contract(threading = ThreadingBehavior.IMMUTABLE_CONDITIONAL)
public class DefaultHttpClientConnectionOperator implements HttpClientConnectionOperator {

    static final String SOCKET_FACTORY_REGISTRY = "http.socket-factory-registry";

    private final Log log = LogFactory.getLog(getClass());

    private final Lookup socketFactoryRegistry;
    private final SchemePortResolver schemePortResolver;
    private final DnsResolver dnsResolver;

    public DefaultHttpClientConnectionOperator(
            final Lookup socketFactoryRegistry,
            final SchemePortResolver schemePortResolver,
            final DnsResolver dnsResolver) {
        super();
        Args.notNull(socketFactoryRegistry, "Socket factory registry");
        this.socketFactoryRegistry = socketFactoryRegistry;
        this.schemePortResolver = schemePortResolver != null ? schemePortResolver :
            DefaultSchemePortResolver.INSTANCE;
        this.dnsResolver = dnsResolver != null ? dnsResolver :
            SystemDefaultDnsResolver.INSTANCE;
    }

    @Override
    public void connect(
            final ManagedHttpClientConnection conn,
            final HttpHost host,
            final InetSocketAddress localAddress,
            final int connectTimeout,
            final SocketConfig socketConfig,
            final HttpContext context) throws IOException {
        final Lookup registry = getSocketFactoryRegistry(context);
        final ConnectionSocketFactory sf = registry.lookup(host.getSchemeName());
        if (sf == null) {
            throw new UnsupportedSchemeException(host.getSchemeName() +
                    " protocol is not supported");
        }
        final InetAddress[] addresses = host.getAddress() != null ?
                new InetAddress[] { host.getAddress() } : this.dnsResolver.resolve(host.getHostName());
        final int port = this.schemePortResolver.resolve(host);
        for (int i = 0; i < addresses.length; i++) {
            final InetAddress address = addresses[i];
            final boolean last = i == addresses.length - 1;

            Socket sock = sf.createSocket(context);
            sock.setSoTimeout(socketConfig.getSoTimeout());
            sock.setReuseAddress(socketConfig.isSoReuseAddress());
            sock.setTcpNoDelay(socketConfig.isTcpNoDelay());
            sock.setKeepAlive(socketConfig.isSoKeepAlive());
            if (socketConfig.getRcvBufSize() > 0) {
                sock.setReceiveBufferSize(socketConfig.getRcvBufSize());
            }
            if (socketConfig.getSndBufSize() > 0) {
                sock.setSendBufferSize(socketConfig.getSndBufSize());
            }

            final int linger = socketConfig.getSoLinger();
            if (linger >= 0) {
                sock.setSoLinger(true, linger);
            }
            conn.bind(sock);

            final InetSocketAddress remoteAddress = new InetSocketAddress(address, port);
            if (this.log.isDebugEnabled()) {
                this.log.debug("Connecting to " + remoteAddress);
            }
            try {
                sock = sf.connectSocket(
                        connectTimeout, sock, host, remoteAddress, localAddress, context);
                conn.bind(sock);
                if (this.log.isDebugEnabled()) {
                    this.log.debug("Connection established " + conn);
                }
                return;
            } catch (final SocketTimeoutException ex) {
                if (last) {
                    throw new ConnectTimeoutException(ex, host, addresses);
                }
            } catch (final ConnectException ex) {
                if (last) {
                    final String msg = ex.getMessage();
                    throw "Connection timed out".equals(msg)
                                    ? new ConnectTimeoutException(ex, host, addresses)
                                    : new HttpHostConnectException(ex, host, addresses);
                }
            } catch (final NoRouteToHostException ex) {
                if (last) {
                    throw ex;
                }
            }
            if (this.log.isDebugEnabled()) {
                this.log.debug("Connect to " + remoteAddress + " timed out. " +
                        "Connection will be retried using another IP address");
            }
        }
    }

    //......
}    
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100

DefaultHttpClientConnectionOperator的connect方法会通过dnsResolver.resolve解析host

小结

HttpClient提供了DnsResolver接口，可以用于自定义DNS解析，是除了使用sun.net.spi.nameservice.provider.去自定义JDK全局dns解析外的另外一种方案。