Skip to main content

Posts

Showing posts with the label html

Parsing HTML with JSoup

The transit demo site I'm building uses data from several websites - such as the Port Authority's press page. Unfortunately, most sites aren't designed for easy parsing; usually data is only available as HTML. So I need to parse HTML to uncover useful information, and if you've ever looked at the source of an average web page, writing a parser from scratch is a challenging task. Enter the jsoup library. This programming interface exposes the structure of a HTML document in a way that can be queried, traversed and even altered by Java programs. I had previously used it to clean out HTML tags from user-submitted content, but recently have been taking advantage of its powerful query engine, which is similar to the way jQuery selectors operate. Let's say I'm parsing some markup that repeats for different stories: I would want to parse this by first finding all of the stories using the <div class="story"> element, and then for each story, quer...