Iterator handling error

556 views
Skip to first unread message

Dmitriy P

unread,
Aug 29, 2024, 8:41:47 AMAug 29
to golang-nuts
What is the best way to handle errors with iterators?

How to handle error when we have some object and some type paginates data (e.g. database/sql.Rows or https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/s3#ListObjectsV2Paginator)?
I found three, but which should be preferable or maybe I've missed some?

```go
type Object smth.Smth
type Paginator interface {
GetNext(ctx) ([]Object, error) // func doing heavy request over internet
HaveNext() bool
}
```

```go
func Iter(ctx, data) iter.Seq2[Object, error]

for obj, err := Iter(...) {
if err != nil {
return err
}
// do smth with Object
}
```

```go
func IterWithErr(ctx, data, err *error) iter.Seq[Object]

var err error
for obj := IterWithErr(..., &err) {
// do smth with Object
}
if err != nil {
return err
}
```

```go
func IterOverIter(ctx, data) iter.Seq2[iter.Seq[Object], error]

for page, err := IterOverIter(...) {
if err != nil {
return err
}
for obj := page(...) {
// do smth with Object
}
}
```


Christoph Berger

unread,
Aug 30, 2024, 6:05:33 AMAug 30
to golang-nuts
Opinionated answer: If a container type cannot iterate over its elements without fail, I would refrain from trying to build a standard iterator for this container type. You'd be shoehorning an iterator that can fail into the iter API that is not designed for dealing with errors. 

Putting half-witted opinions aside, this thread in /r/golang might provide some food for thought. If you want to go with one of your three options, I'd favor the first one for its simplicity. It seems to work, and I don't see how the other two improve over the first one. 

Bob Glickstein

unread,
Aug 30, 2024, 12:55:04 PMAug 30
to golang-nuts
I like:

```go
func Iter(ctx, data) (iter.Seq[Object], *error)
```

There are some examples of this pattern in https://pkg.go.dev/github.com/bobg/seqs

Cheers,
- Bob

gbarr

unread,
Aug 31, 2024, 7:38:53 AMAug 31
to golang-nuts
I would likely go with something similar to how you would currently use bufio.Scanner but comine the use of  .Scan()  and .Text() as Range()

```go
iter := rows.Iter(ctx)
for obj := iter.Range {
    // do something with obj
}
if err := iter.Err(); err != nil {
    return err
}
```

Or if rows can only have a single active iterator the there is no need for the iter object

```go
for obj := rows.Range(ctx) {
    // do something with obj
}
if err := rows.Err(); err != nil {
    return err
}
```

Graham.

Christoph Berger

unread,
Sep 3, 2024, 1:19:31 PMSep 3
to golang-nuts
I like this approach.
  • bufio.Scanner-style errors are a known pattern from the stdlib
  • The approach keeps the key and value params free for their intended use
  • It doesn't require nested iter seqs
  • It is compatible with range loops
Made some sample code as PoC -> https://go.dev/play/p/6qhH6ZN6f0S (Doesn't work in the playground due to network restrictions)

cpasmaboiteaspam

unread,
Sep 5, 2024, 7:11:39 PMSep 5
to golang-nuts
Hello,

I would definitely use func Iter(ctx, data) iter.Seq2[T, error]

Because its standard and it is easy for the end user to build a bridge to a scanner like API 
such as struct[T any]{ All() iter.Seq[T]; Err() error },
or anything else.

Because i have make the trailing error return a first citizen
that signature fits straight into my own experiments .

For certain, i would not use func IterWithErr(ctx, data, err *error) iter.Seq[T]
It data raced the second i tried this approach.

spaghetti for thoughts.

alex-coder

unread,
Sep 9, 2024, 4:46:15 AMSep 9
to golang-nuts
Hi,
Java is certainly not Go, now we have a great simplification to develop code, but you can clarify the context for making a decision in the description of the ValueListHandler J2EE pattern.

Regards,

пятница, 6 сентября 2024 г. в 02:11:39 UTC+3, cpasmaboiteaspam:
Reply all
Reply to author
Forward
0 new messages