html parser

[Marcus Rohrmoser]

what html parser can you recommend? I come from https://aantron.github.io/lambdasoup/ and liked it, but want to port some stuff to lua now.

Cheers,
Marcus

[Martin Eden]

that ocaml code looks nice, why you want lua?

-- Martin

[Marcus Rohrmoser]

On Fri, 31 Jul 2026 18:11:36 +0200
"'Martin Eden' via lua-l" <lu...@googlegroups.com> wrote:

> On 2026-07-30 20:07, Marcus Rohrmoser wrote:

> > https://aantron.github.io/lambdasoup/

> >
> that ocaml code looks nice, why you want lua?

it's indeed beautiful and I immensely enjoy it[1], but while exploring convivial, malleable tools, I consider OCaml too heavy a toolchain - compared to the lua runtime.

I want tinkering with tools approachable for hobbyists, newcomers and brave non-techs.

Maintaining an OCaml toolchain (in fact any toolchain) feels too high a bar for what I am after.

Cheers,
Marcus

[1]: e.g. at https://codeberg.org/mro/pagerake/src/commit/9d5bf87/footnote.ml#L85 or https://codeberg.org/mro/uf2ics/src/commit/92e86af5/uf2.ml#L52

[Martin Eden]

I like data processing and creating _tools for data processing_.
Lua is nice for this activity. It's small and complete.

But here is no big hyped libraries for this. We have LPEG (if you like
C implementations, metamethods and it's names in general). Here is
stock-like regexps (I dislike regexps).

But you can write your own for yourself if you want to implement _tool_
instead of doing one-time job fast. For example (1) that's how my code
for parsing Lua grammar looks like.

Or look at the LISP direction if you want to dive into your data structure.

[1]:
https://github.com/martin-eden/workshop/blob/master/concepts/lua/syntax/expression.lua

-- Martin

[Sean Conner]

It was thus said that the Great Marcus Rohrmoser once stated:

>
> it's indeed beautiful and I immensely enjoy it[1], but while exploring
> convivial, malleable tools, I consider OCaml too heavy a toolchain -
> compared to the lua runtime.
>
> I want tinkering with tools approachable for hobbyists, newcomers and
> brave non-techs.

I do have an HTML parser [2] in LPeg, about 500 lines of code. I wouldn't
consider LPeg all that approachable for hobbyists, newcomers or even brave
non-techs. I like it, but it took me about a year of working with it to
become comfortable, and a few more before I understood all the oeprators and
captures one can make. And even the simpler re module requires
understanding both BNF and the concepts from the lpeg module.

Here's a sample of the LPeg parser:

local parse_tags = P {
-- ...
A = tagi('a' , A_attr , (V'inline' - V'A')^0),
IMG = tagi('img' , IMG_attr , EMPTY,true),
-- ...
}

I later swapped the LPeg-based HTML parser for one written with peg (a C
tool [3]) that I modified [4] not because of speed, but because LPeg was
consuming more memory than I was comfortable with. That peg was faster was
a benefit, but again, peg is a parser generator, something that might not be
approachable for hobbyists, newcomings or brave non-techs.

Here's a sample of the C peg code:

A <- "<" _A & ET { tagi(yy,"a"); } A_attr* S* ">" (!A inline)* "</" _A ">" { endtag(yy); }
IMG <- "<" _IMG & ET { tagi(yy,"img"); } IMG_attr* S* ">" (S* "</" _IMG ">")? { endtag(yy); }

# ...

# my customized syntax, using a backtick, not a single quote
# for insensitive comparison.

_A <- `a`
_IMG <- `img`

# othewise, it was

_A <- [Aa]
_IMG <- [Ii][Mm][Gg]

I personally find the peg version to be easier to read than the LPeg
version, but I can't say the same for other people.

-spc

[1] Not my footnote.

[2] HTML4 strict, which is what I needed. I don't have an HTML5 parser
because the "living standard" changes too much, and the "grammar"
(such as it is) is pages and pages of English text describing in
minute detail how one should parse HTML5 character by character.
Ick.

[3] As tools go, it's fairly simple. A single executable that takes PEG
input and outputs C source which you need to compile. It's similar
in struture to lex and yacc.

[4] I modified the C peg library to make case insensitive comparrisons
easier to write, and faster to parse.

[Aaron B.]

On Thu, 30 Jul 2026 20:07:28 +0200
Marcus Rohrmoser <me....@mro.name> wrote:

>
> what html parser can you recommend? I come from https://aantron.github.io/lambdasoup/ and liked it, but want to port some stuff to lua now.
>

Not entirely sure if this is what you want: It provides a parsed HTML
object manipulated with a Javascript-like DOM API, as opposed to a
simple tree-like structure of the data:

https://gitlab.com/craigbarnes/lua-gumbo

I've had good enough results with it for my purposes installing it with
Luarocks, but admit the entire DOM thing is over my head sometimes.

--
Aaron B. <aa...@zadzmo.org>

[Marcus Rohrmoser]

On Sat, 1 Aug 2026 15:52:25 -0400
Sean Conner <se...@conman.org> wrote:

> I do have an HTML parser [2] in LPeg, about 500 lines of code. I wouldn't
> consider LPeg all that approachable for hobbyists, newcomers or even brave
> non-techs.

maybe I won't parse HTML in its entirety and flavours, but run a filter on text-level, sift the limited tags I care about[1], and amend the output.

I still hesitate, but may turn to LPEGs (rather than a homegrown state-machine) to be a more idiomatic and maybe hone the craft.

Thanks for your input,
Marcus

[1]: at first something like https://codeberg.org/mro/pagerake/src/commit/9d5bf870/prake.ml#L35-L41

[TopchetoEU]

I have written my own HTML (and XML) parser and have used it with great success to scrape websites (as complex as instagram and reddit). I can't vouch for its correctness, nor its efficiency, as I never bothered to write a test suite. However, it has served me well...

https://git.topcheto.eu/tal/ref/master/files/lib/std/fmt/xml.lua

It also has serviceable query functions for the nodes, so you can traverse the tree easier. Not documented, so use at your own (sanity's) risk.

> --
> You received this message because you are subscribed to the Google Groups "lua-l" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to lua-l+un...@googlegroups.com.
> To view this discussion visit https://groups.google.com/d/msgid/lua-l/20260730200728.b97e4cd0f6f8c26108595a66%40mro.name.
>