There is a scanner in pkg/go/scanner/scanner.go.
You should pay special attention to the following:
// Read the next Unicode char into S.ch.
// S.ch < 0 means end-of-file.
//
func (S *Scanner) next() {
if S.offset < len(S.src) {
S.pos.Offset = S.offset;
S.pos.Column++;
r, w := int(S.src[S.offset]), 1;
switch {
case r == '\n':
S.pos.Line++;
S.pos.Column = 0;
case r >= 0x80:
// not ASCII
r, w = utf8.DecodeRune(S.src[S.offset:len(S.src)])
}
S.offset += w;
S.ch = r;
} else {
S.pos.Offset = len(S.src);
S.ch = -1; // eof
}
}
You might also want to use "expect" method, which calls errorHandler
if next char is not expected, also there are isLetter and isDigit
declared; scanMantissa is the simplest scanner example, which will go
through all digits on given constraint - input is "base" variable, if
you set it to 10, then it goes over all digits 0-9 and returns (next
scan command will start from where this one finished). You can use
S.pos afterwards to create a slice.
In case the whole thing is too much, I think that this function here
does the job - go through a line with it, then use slice and do with
the resulting string slice what you want. You must release the string
if you don't want to keep whole file hidden in memory afterwards,
slice is a string pointer.