A Full-Fledged Java-Based HTTP Client

1. Context

We are going to write a mini-browser that can establish a connection with any website (URL) over HTTP protocol and then send requests & receive response messages. For example, you may hit https://google.com and receive the HTML content or hit a REST API endpoint and receive JSON content. We use plain Java and do not use any third-party library.

2. Approach

java.net package provides interfaces and classes to establish a communication link between client and server over the network. Following steps are involved when java.net.HttpURLConnection is used for communication over HTTP:

1. Create a URL object
2. Call openConnection() on URL to get a connection
3. Add necessary connection properties and request headers
4. Establish the actual connection by calling connect() method on HttpURLConnection
5. Read response headers from the connection
6. Read the response as input stream obtained from the connection

3. Send Request and Receive Response

The sendRequest() method sequentially calls methods to create URL, get Http URL connection, set up connection properties, add request headers, establish an actual connection, and finally receive HTTP response.

/**
 * Opens a connection and sends Http request to given URL.
 * @param url URL to be invoked.
 * @return Response as a stream.
 */
private InputStream sendRequest(String url) {
    try {
        URL urlInstance = new URL(url);
        HttpURLConnection connection = (HttpURLConnection) urlInstance.
                openConnection();
        // Set timeout in milliseconds
        connection.setConnectTimeout(20000);
        // Set request headers
        addRequestHeaders(connection);
        // Establiish a connection
        connection.connect();
        // Read response status and headers
        readResponseStatusAndHeaders(connection);
        // Return response
        return connection.getInputStream();
    } catch (IOException ex) {
        ex.printStackTrace();
    }
    return null;
}

The following method adds HTTP request headers to the connection. Get the complete list of request headers from here.

/**
 * Adds request headers to connection.
 * @param connection Http URL connection instance.
 */
private void addRequestHeaders(HttpURLConnection connection) {
    connection.addRequestProperty("Accept", "text/plain");
    connection.addRequestProperty("Accept-Language", "en");
    connection.addRequestProperty("Cache-Control", "no-cache");
}

The following method extracts the response code, response message, and response headers from the connection. Get the complete list of response headers from here.

/**
 * Reads response code, message and header fields.
 * @param connection Http URL connection instance.
 * @throws IOException If an error occrred while connecting to server.
 */
private void readResponseStatusAndHeaders(HttpURLConnection connection) throws
        IOException {
    System.out.println("Response code: " + connection.getResponseCode());
    System.out.println(
            "Response message: " + connection.getResponseMessage());
    System.out.println("Response header: Content-Length: " + connection.
            getHeaderField("Content-Length"));
}

4. Read Response

Finally, extract the response data from the input stream obtained from the connection.

/**
 * Prints the response to console.
 * @param response Response stream.
 */
private void processResponse(InputStream response) {
    if (response == null) {
        System.out.println("No or blank response received");
    } else {
        StringBuilder data = new StringBuilder();
        try {
            int bufferSize = 2048;
            byte[] inputData = new byte[bufferSize];
            int count = 0;
            while ((count = response.read(inputData, 0, bufferSize)) != -1) {
                data.append(new String(inputData, 0, count, "UTF-8"));
            }
        } catch (IOException ex) {
            ex.printStackTrace();
        }
        System.out.println("Data received: " + data.toString());
    }
}

5. Run the Program

Here is the main method to test the HttpClient. It accepts the URL as string input from the user.

/**
 * Main method of the program.
 * @param args Input arguments to the program.
 */
public static void main(String[] args) {
    HttpClient handler = new HttpClient(false);
    Scanner scanner = new Scanner(System.in);
    while(true) {
        System.out.println("URL (q to quit):");
        String url = scanner.nextLine();
        if(!url.equalsIgnoreCase("q")) {
            InputStream response = handler.sendRequest(url.trim());
            handler.processResponse(response);
        } else {
            break;
        }
    }
}

6. Conclusion

You may download the complete source code from here: HttpClient.java. For a sophisticated and efficient HttpClient use Apache HttpClient.