Extract All Links of a web page in Java using jsoup

Tags: jsoup HTML Parser

In this post, we show you how to extract all links from a web page using jsoup Java library.

Add jsoup library to your Java project

To use jsoup Java library in the Gradle build project, add the following dependency into the build.gradle file. 

compile 'org.jsoup:jsoup:1.13.1'

To use jsoup Java library in the Maven build project, add the following dependency into the pom.xml file. 

<dependency>
    <groupId>org.jsoup</groupId>
    <artifactId>jsoup</artifactId>
    <version>1.13.1</version>
</dependency>

To download the jsoup-1.13.1.jar file you can visit jsoup download page at jsoup.org/download

In jsoup library to get href’s value of anchor tag we can use Element.attr() method.

  • Element.attr(“href”) method to get relative URL
  • Element.attr(“abs:href”) method to get absolute URL

Example 1 using Document.getElementsByTag() method to get links Elements 

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

import java.io.IOException;

public class GetAllLinkExample1 {
    public static void main(String[] args) {
        try {
            String url = "https://simplesolution.dev";
            Document document = Jsoup.connect(url).get();

            Elements allLinks = document.getElementsByTag("a");

            for(Element link: allLinks) {
                String relativeUrl = link.attr("href");
                String absoluteUrl = link.attr("abs:href");

                System.out.println("Relative URL: " + relativeUrl);
                System.out.println("Absolute URL: " + absoluteUrl);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
Output: 
Relative URL: /page/2/
Absolute URL: https://simplesolution.dev/page/2/
Relative URL: /page/3/
Absolute URL: https://simplesolution.dev/page/3/

Example 2 using Document.select() method to get links Elements 

import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;

import java.io.IOException;

public class GetAllLinkExample2 {
    public static void main(String[] args) {
        try {
            String url = "https://simplesolution.dev";
            Document document = Jsoup.connect(url).get();

            Elements allLinks = document.select("a[href]");

            for(Element link: allLinks) {
                String relativeUrl = link.attr("href");
                String absoluteUrl = link.attr("abs:href");

                System.out.println("Relative URL: " + relativeUrl);
                System.out.println("Absolute URL: " + absoluteUrl);
            }
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
Output: 
Relative URL: /page/2/
Absolute URL: https://simplesolution.dev/page/2/
Relative URL: /page/3/
Absolute URL: https://simplesolution.dev/page/3/

Happy Coding 😊

Pretty Print XML String and XML File in Java using jsoup

jsoup extract CSS class name of HTML element in Java

jsoup extract ID and name of HTML element in Java

jsoup extract text and attributes of HTML element in Java

jsoup extract Inner and Outer HTML of HTML Element in Java

jsoup extract JavaScript from HTML script element in Java

jsoup extract custom data attributes of HTML5 Element in Java

jsoup extract Website Title in Java

jsoup parse HTML Document from a Java String

jsoup parse HTML Document from an URL in Java

jsoup parse HTML Document from a File and InputStream in Java

Pretty Printing HTML String in Java using jsoup

jsoup Get HTML elements by CSS class name in Java

Clean HTML String to get Safe HTML from Untrusted HTML in Java using jsoup

jsoup Get All HTML Elements in Java