Using JSoup to parse a html string with Clojure, the source as the following
Dependencies
:dependencies [[org.clojure/clojure "1.10.1"]
[org.jsoup/jsoup "1.13.1"]]
Source code
(require '[clojure.string :as str])
(def HTML (str "<html><head><title>Website title</title></head>
<body><p>Sample paragraph number 1 </p>
<p>Sample paragraph number 2</p>
</body></html>"))
(defn fetch_html [html]
(let [soup (Jsoup/parse html)
titles (.title soup)
paragraphs (.getElementsByTag soup "p")]
{:title titles :paragraph paragraphs}))
(fetch_html HTML)
Expected result
{:title "Website title",
:paragraph ["Sample paragraph number 1"
"Sample paragraph number 2"]}
Unfortunately, the result is not as expected
user ==> (fetch_html HTML)
{:title "Website title", :paragraph []}
Advertisement
Answer
(.getElementsByTag …) returns a sequence of Element’s, you need to call .text() method on each element to get the text value. I’m using Jsoup ver 1.13.1.
(ns core
(:import (org.jsoup Jsoup))
(:require [clojure.string :as str]))
(def HTML (str "<html><head><title>Website title</title></head>
<body><p>Sample paragraph number 1 </p>
<p>Sample paragraph number 2</p>
</body></html>"))
(defn fetch_html [html]
(let [soup (Jsoup/parse html)
titles (.title soup)
paragraphs (.getElementsByTag soup "p")]
{:title titles :paragraph (mapv #(.text %) paragraphs)}))
(fetch_html HTML)
Also consider using Reaver, which is a Clojure library that wraps JSoup, or any other wrappers like others have suggested.