[go] fmt: require newlines to match when scanning with a format
382 views
Skip to first unread message
Rob Pike (Gerrit)
Jun 5, 2015, 5:27:42 PM6/5/15
to Ian Lance Taylor, Rob Pike, golang-co...@googlegroups.com
Rob Pike uploaded a change:
https://go-review.googlesource.com/10779
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 89 insertions(+), 14 deletions(-)
diff --git a/src/fmt/doc.go b/src/fmt/doc.go
index cbca6ab..c3952d7 100644
--- a/src/fmt/doc.go
+++ b/src/fmt/doc.go
@@ -239,9 +239,15 @@
Fscanf and Fscanln read from a specified io.Reader; Sscan,
Sscanf and Sscanln read from an argument string. Scanln,
Fscanln and Sscanln stop scanning at a newline and require that
- the items be followed by one; Scanf, Fscanf and Sscanf require
- newlines in the input to match newlines in the format; the other
- routines treat newlines as spaces.
+ the items be followed by one.
+
+ Scanf, Fscanf and Sscanf require that (after skipping spaces) newlines
+ in the format are matched by newlines in the input and vice versa.
+ This behavior differs from the corresponding routines in C, which
+ uniformly treat newlines as spaces.
+
+ The routines that have no format - Scan, Fscan, Sscan - treat newlines
+ in the input as spaces.
Scanf, Fscanf, and Sscanf parse the arguments according to a
format string, analogous to that of Printf. For example, %x
diff --git a/src/fmt/scan.go b/src/fmt/scan.go
index 9572530..609674e 100644
--- a/src/fmt/scan.go
+++ b/src/fmt/scan.go
@@ -1074,12 +1074,13 @@
}
// advance determines whether the next characters in the input match
-// those of the format. It returns the number of bytes (sic) consumed
-// in the format. Newlines included, all runs of space characters in
-// either input or format behave as a single space. This routine also
-// handles the %% case. If the return value is zero, either format
-// starts with a % (with no following %) or the input is empty.
-// If it is negative, the input did not match the string.
+// those of the format. It returns the number of bytes (sic) consumed
+// in the format. All runs of space characters in either input or
+// format behave as a single space. Newlines are special, though:
+// they must match in format and in input. This routine also handles
+// the %% case. If the return value is zero, either format starts
+// with a % (with no following %) or the input is empty. If it is
+// negative, the input did not match the string.
func (s *ss) advance(format string) (i int) {
for i < len(format) {
fmtc, w := utf8.DecodeRuneInString(format[i:])
@@ -1092,24 +1093,39 @@
i += w // skip the first %
}
sawSpace := false
+ wasNewline := false
for isSpace(fmtc) && i < len(format) {
+ wasNewline = fmtc == '\n'
sawSpace = true
i += w
fmtc, w = utf8.DecodeRuneInString(format[i:])
}
if sawSpace {
- // There was space in the format, so there should be space (EOF)
+ // There was space in the format, so there should be space
// in the input.
inputc := s.getRune()
- if inputc == eof || inputc == '\n' {
- // If we've reached a newline, stop now; don't read ahead.
+ if inputc == eof {
return
}
if !isSpace(inputc) {
- // Space in format but not in input: error
+ // Space in format but not in input.
s.errorString("expected space in input to match format")
}
- s.skipSpace(true)
+ // Skip spaces but stop at newline.
+ for inputc != '\n' && isSpace(inputc) {
+ inputc = s.getRune()
+ }
+ if inputc == '\n' {
+ if !wasNewline {
+ s.errorString("newline in input does not match format")
+ }
+ // We've reached a newline, stop now; don't read further.
+ return
+ }
+ s.UnreadRune()
+ if wasNewline {
+ s.errorString("newline in format does not match input")
+ }
continue
}
inputc := s.mustReadRune()
diff --git a/src/fmt/scan_test.go b/src/fmt/scan_test.go
index a932831..e51be07 100644
--- a/src/fmt/scan_test.go
+++ b/src/fmt/scan_test.go
@@ -1044,3 +1044,56 @@
t.Errorf("odd count: got count, err = %d, %v; expected 0, error", n, err)
}
}
+
+// Issue 10862, part 1.
+func TestScanNewlinesAreSpaces(t *testing.T) {
+ var a, b int
+ n, err := Sscan("1\n2", &a, &b)
+ if n != 2 {
+ t.Errorf("expected to scan 2 items, scanned %d", n)
+ }
+ if err != nil {
+ t.Error(err)
+ }
+}
+
+// Issue 10862, part 2.
+func TestScanfNewlinesAreNotSpaces(t *testing.T) {
+ var a, b int
+ n, err := Sscanf("1\n2", "%d %d", &a, &b)
+ if n != 1 {
+ t.Errorf("newline in input: expected to scan 1 item, scanned %d", n)
+ }
+ if err == nil {
+ t.Error("newline in input: expected error")
+ }
+ n, err = Sscanf("1 \n2", "%d %d", &a, &b)
+ if n != 1 {
+ t.Errorf("space-newline in input: expected to scan 1 item, scanned %d",
n)
+ }
+ if err == nil {
+ t.Error("space-newline in input: expected error")
+ }
+ n, err = Sscanf("1 2", "%d\n%d", &a, &b)
+ if n != 1 {
+ t.Errorf("newline in format: expected to scan 1 item, scanned %d", n)
+ }
+ if err == nil {
+ t.Error("newline in format: expected error")
+ }
+ n, err = Sscanf("1 2", "%d \n%d", &a, &b)
+ if n != 1 {
+ t.Errorf("space-newline in format: expected to scan 1 item, scanned %d",
n)
+ }
+ if err == nil {
+ t.Error("space-newline in format: expected error")
+ }
+ // When they line up it should work.
+ n, err = Sscanf("1 \n2", "%d \n%d", &a, &b)
+ if n != 2 {
+ t.Errorf("space-newline in format: expected to scan 2 items,
scanned %d", n)
+ }
+ if err != nil {
+ t.Error("space-newline in format: ", err)
+ }
+}
--
https://go-review.googlesource.com/10779
https://go-review.googlesource.com/10779
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 89 insertions(+), 14 deletions(-)
diff --git a/src/fmt/doc.go b/src/fmt/doc.go
index cbca6ab..c3952d7 100644
--- a/src/fmt/doc.go
+++ b/src/fmt/doc.go
@@ -239,9 +239,15 @@
Fscanf and Fscanln read from a specified io.Reader; Sscan,
Sscanf and Sscanln read from an argument string. Scanln,
Fscanln and Sscanln stop scanning at a newline and require that
- the items be followed by one; Scanf, Fscanf and Sscanf require
- newlines in the input to match newlines in the format; the other
- routines treat newlines as spaces.
+ the items be followed by one.
+
+ Scanf, Fscanf and Sscanf require that (after skipping spaces) newlines
+ in the format are matched by newlines in the input and vice versa.
+ This behavior differs from the corresponding routines in C, which
+ uniformly treat newlines as spaces.
+
+ The routines that have no format - Scan, Fscan, Sscan - treat newlines
+ in the input as spaces.
Scanf, Fscanf, and Sscanf parse the arguments according to a
format string, analogous to that of Printf. For example, %x
diff --git a/src/fmt/scan.go b/src/fmt/scan.go
index 9572530..609674e 100644
--- a/src/fmt/scan.go
+++ b/src/fmt/scan.go
@@ -1074,12 +1074,13 @@
}
// advance determines whether the next characters in the input match
-// those of the format. It returns the number of bytes (sic) consumed
-// in the format. Newlines included, all runs of space characters in
-// either input or format behave as a single space. This routine also
-// handles the %% case. If the return value is zero, either format
-// starts with a % (with no following %) or the input is empty.
-// If it is negative, the input did not match the string.
+// those of the format. It returns the number of bytes (sic) consumed
+// in the format. All runs of space characters in either input or
+// format behave as a single space. Newlines are special, though:
+// they must match in format and in input. This routine also handles
+// the %% case. If the return value is zero, either format starts
+// with a % (with no following %) or the input is empty. If it is
+// negative, the input did not match the string.
func (s *ss) advance(format string) (i int) {
for i < len(format) {
fmtc, w := utf8.DecodeRuneInString(format[i:])
@@ -1092,24 +1093,39 @@
i += w // skip the first %
}
sawSpace := false
+ wasNewline := false
for isSpace(fmtc) && i < len(format) {
+ wasNewline = fmtc == '\n'
sawSpace = true
i += w
fmtc, w = utf8.DecodeRuneInString(format[i:])
}
if sawSpace {
- // There was space in the format, so there should be space (EOF)
+ // There was space in the format, so there should be space
// in the input.
inputc := s.getRune()
- if inputc == eof || inputc == '\n' {
- // If we've reached a newline, stop now; don't read ahead.
+ if inputc == eof {
return
}
if !isSpace(inputc) {
- // Space in format but not in input: error
+ // Space in format but not in input.
s.errorString("expected space in input to match format")
}
- s.skipSpace(true)
+ // Skip spaces but stop at newline.
+ for inputc != '\n' && isSpace(inputc) {
+ inputc = s.getRune()
+ }
+ if inputc == '\n' {
+ if !wasNewline {
+ s.errorString("newline in input does not match format")
+ }
+ // We've reached a newline, stop now; don't read further.
+ return
+ }
+ s.UnreadRune()
+ if wasNewline {
+ s.errorString("newline in format does not match input")
+ }
continue
}
inputc := s.mustReadRune()
diff --git a/src/fmt/scan_test.go b/src/fmt/scan_test.go
index a932831..e51be07 100644
--- a/src/fmt/scan_test.go
+++ b/src/fmt/scan_test.go
@@ -1044,3 +1044,56 @@
t.Errorf("odd count: got count, err = %d, %v; expected 0, error", n, err)
}
}
+
+// Issue 10862, part 1.
+func TestScanNewlinesAreSpaces(t *testing.T) {
+ var a, b int
+ n, err := Sscan("1\n2", &a, &b)
+ if n != 2 {
+ t.Errorf("expected to scan 2 items, scanned %d", n)
+ }
+ if err != nil {
+ t.Error(err)
+ }
+}
+
+// Issue 10862, part 2.
+func TestScanfNewlinesAreNotSpaces(t *testing.T) {
+ var a, b int
+ n, err := Sscanf("1\n2", "%d %d", &a, &b)
+ if n != 1 {
+ t.Errorf("newline in input: expected to scan 1 item, scanned %d", n)
+ }
+ if err == nil {
+ t.Error("newline in input: expected error")
+ }
+ n, err = Sscanf("1 \n2", "%d %d", &a, &b)
+ if n != 1 {
+ t.Errorf("space-newline in input: expected to scan 1 item, scanned %d",
n)
+ }
+ if err == nil {
+ t.Error("space-newline in input: expected error")
+ }
+ n, err = Sscanf("1 2", "%d\n%d", &a, &b)
+ if n != 1 {
+ t.Errorf("newline in format: expected to scan 1 item, scanned %d", n)
+ }
+ if err == nil {
+ t.Error("newline in format: expected error")
+ }
+ n, err = Sscanf("1 2", "%d \n%d", &a, &b)
+ if n != 1 {
+ t.Errorf("space-newline in format: expected to scan 1 item, scanned %d",
n)
+ }
+ if err == nil {
+ t.Error("space-newline in format: expected error")
+ }
+ // When they line up it should work.
+ n, err = Sscanf("1 \n2", "%d \n%d", &a, &b)
+ if n != 2 {
+ t.Errorf("space-newline in format: expected to scan 2 items,
scanned %d", n)
+ }
+ if err != nil {
+ t.Error("space-newline in format: ", err)
+ }
+}
--
https://go-review.googlesource.com/10779
Rob Pike (Gerrit)
Jun 5, 2015, 5:35:42 PM6/5/15
to Rob Pike, golang-co...@googlegroups.com
Rob Pike has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1: Run-TryBot+1
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Gerrit-HasComments: No
fmt: require newlines to match when scanning with a format
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Gerrit-HasComments: No
Gobot Gobot (Gerrit)
Jun 5, 2015, 5:36:22 PM6/5/15
to Rob Pike, golang-co...@googlegroups.com
Gobot Gobot has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1:
TryBots beginning. Status page: http://farmer.golang.org/try?commit=aaf9ed07
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
fmt: require newlines to match when scanning with a format
TryBots beginning. Status page: http://farmer.golang.org/try?commit=aaf9ed07
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gobot Gobot (Gerrit)
Jun 5, 2015, 5:40:44 PM6/5/15
to Rob Pike, golang-co...@googlegroups.com
Gobot Gobot has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1:
This change failed on linux-amd64:
See
https://storage.googleapis.com/go-build-log/aaf9ed07/linux-amd64_0db9f942.log
Consult https://build.golang.org/ to see whether it's a new failure. Other
builds still in progress; subsequent failure notices suppressed until final
report.
Gobot Gobot (Gerrit)
Jun 5, 2015, 5:52:09 PM6/5/15
to Rob Pike, golang-co...@googlegroups.com
Gobot Gobot has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1: TryBot-Result-1
6 of 20 TryBots failed:
Failed on linux-amd64:
https://storage.googleapis.com/go-build-log/aaf9ed07/linux-amd64_0db9f942.log
Failed on linux-amd64-race:
https://storage.googleapis.com/go-build-log/aaf9ed07/linux-amd64-race_cd6456fa.log
Failed on linux-386:
https://storage.googleapis.com/go-build-log/aaf9ed07/linux-386_af23d618.log
Failed on windows-386-gce:
https://storage.googleapis.com/go-build-log/aaf9ed07/windows-386-gce_d1cd5284.log
Failed on windows-amd64-gce:
https://storage.googleapis.com/go-build-log/aaf9ed07/windows-amd64-gce_59964da0.log
Failed on openbsd-386-gce56:
https://storage.googleapis.com/go-build-log/aaf9ed07/openbsd-386-gce56_bce52080.log
Consult https://build.golang.org/ to see whether they are new failures.
Aamir Khan (Gerrit)
Jun 6, 2015, 1:34:14 PM6/6/15
to Rob Pike, Gobot Gobot, golang-co...@googlegroups.com
Aamir Khan has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1:
I added a few more test cases,
n, err = Sscanf("1\n2", "%d \n%d", &a, &b) scans two elements. While,
n, err = Sscanf("1\n2", "%d \n %d", &a, &b) scans just one resulting in
newline error when scanning second element.
I expect both to either result in error or work fine. With the updated
documentation, both should work fine.
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
fmt: require newlines to match when scanning with a format
I added a few more test cases,
n, err = Sscanf("1\n2", "%d \n%d", &a, &b) scans two elements. While,
n, err = Sscanf("1\n2", "%d \n %d", &a, &b) scans just one resulting in
newline error when scanning second element.
I expect both to either result in error or work fine. With the updated
documentation, both should work fine.
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Aamir Khan (Gerrit)
Jun 7, 2015, 12:54:47 PM6/7/15
to Rob Pike, Gobot Gobot, golang-co...@googlegroups.com
Aamir Khan has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1:
(1 comment)
https://go-review.googlesource.com/#/c/10779/1/src/fmt/scan.go
File src/fmt/scan.go:
PS1, Line 1079: Newlines are special, though:
: // they must match in format and in input.
this can be tidied up a bit.
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Rob Pike (Gerrit)
Jun 10, 2015, 11:41:00 AM6/10/15
to Rob Pike, Gobot Gobot, Aamir Khan, golang-co...@googlegroups.com
Reviewers: Gobot Gobot
Rob Pike uploaded a new patch set:
Rob Pike uploaded a new patch set:
https://go-review.googlesource.com/10779
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 87 insertions(+), 14 deletions(-)
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
Rob Pike (Gerrit)
Jun 10, 2015, 11:41:31 AM6/10/15
to Rob Pike, Gobot Gobot, Aamir Khan, golang-co...@googlegroups.com
Rob Pike has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 2: Run-TryBot+1
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Gerrit-HasComments: No
fmt: require newlines to match when scanning with a format
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Gobot Gobot (Gerrit)
Jun 10, 2015, 11:42:26 AM6/10/15
to Rob Pike, Aamir Khan, golang-co...@googlegroups.com
Gobot Gobot has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 2:
TryBots beginning. Status page: http://farmer.golang.org/try?commit=90bf1241
fmt: require newlines to match when scanning with a format
TryBots beginning. Status page: http://farmer.golang.org/try?commit=90bf1241
Rob Pike (Gerrit)
Jun 10, 2015, 11:45:10 AM6/10/15
to Rob Pike, Gobot Gobot, Aamir Khan, golang-co...@googlegroups.com
Reviewers: Gobot Gobot
Rob Pike uploaded a new patch set:
Rob Pike uploaded a new patch set:
https://go-review.googlesource.com/10779
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 87 insertions(+), 14 deletions(-)
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
Gobot Gobot (Gerrit)
Jun 10, 2015, 12:00:13 PM6/10/15
to Rob Pike, Aamir Khan, golang-co...@googlegroups.com
Gobot Gobot has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 2: TryBot-Result+1
TryBots are happy.
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Aamir Khan (Gerrit)
Jun 10, 2015, 12:58:56 PM6/10/15
to Rob Pike, Gobot Gobot, golang-co...@googlegroups.com
Aamir Khan has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 3:
(3 comments)
https://go-review.googlesource.com/#/c/10779/3/src/fmt/scan.go
File src/fmt/scan.go:
Line 38: // unless the scan operation is Scanln, Fscanln or Sscanln, in
which case
newlines are special if format is specified.
Line 44: // characters. Newlines are treated as space unless the scan
operation
same, newlines are special if format is specified.
Line 84: // If that is less than the number of arguments, err will report
why.
say about newline being special here?
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Gerrit-HasComments: Yes
fmt: require newlines to match when scanning with a format
(3 comments)
https://go-review.googlesource.com/#/c/10779/3/src/fmt/scan.go
File src/fmt/scan.go:
Line 38: // unless the scan operation is Scanln, Fscanln or Sscanln, in
which case
newlines are special if format is specified.
Line 44: // characters. Newlines are treated as space unless the scan
operation
same, newlines are special if format is specified.
Line 84: // If that is less than the number of arguments, err will report
why.
say about newline being special here?
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Rob Pike (Gerrit)
Jun 10, 2015, 1:42:07 PM6/10/15
to Rob Pike, Gobot Gobot, Aamir Khan, golang-co...@googlegroups.com
Rob Pike has posted comments on this change.
fmt: require newlines to match when scanning with a format
Line 44: // characters. Newlines are treated as space unless the scan
operation
> same, newlines are special if format is specified.
Done
Line 84: // If that is less than the number of arguments, err will report
why.
> say about newline being special here?
Done
fmt: require newlines to match when scanning with a format
Patch Set 3:
(3 comments)
https://go-review.googlesource.com/#/c/10779/3/src/fmt/scan.go
File src/fmt/scan.go:
Line 38: // unless the scan operation is Scanln, Fscanln or Sscanln, in
which case
> newlines are special if format is specified.
Done
(3 comments)
https://go-review.googlesource.com/#/c/10779/3/src/fmt/scan.go
File src/fmt/scan.go:
Line 38: // unless the scan operation is Scanln, Fscanln or Sscanln, in
which case
> newlines are special if format is specified.
Line 44: // characters. Newlines are treated as space unless the scan
operation
> same, newlines are special if format is specified.
Line 84: // If that is less than the number of arguments, err will report
why.
> say about newline being special here?
Aamir Khan (Gerrit)
Jun 11, 2015, 4:15:53 AM6/11/15
to Rob Pike, Gobot Gobot, golang-co...@googlegroups.com
Aamir Khan has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 3:
seems like you haven't pushed the updated code though
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Gerrit-HasComments: No
fmt: require newlines to match when scanning with a format
seems like you haven't pushed the updated code though
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Rob Pike (Gerrit)
Jun 11, 2015, 10:17:12 AM6/11/15
to Rob Pike, Gobot Gobot, Aamir Khan, golang-co...@googlegroups.com
Reviewers: Gobot Gobot
Rob Pike uploaded a new patch set:
Rob Pike uploaded a new patch set:
https://go-review.googlesource.com/10779
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 94 insertions(+), 19 deletions(-)
fmt: require newlines to match when scanning with a format
The documentation (which has been reworked for clarity but not
changed semantically) claims that Scanf etc. require newlines to
match in input and format, but this was not properly implemented.
This CL makes the code honor the documentation.
Scanning without a format is unaffected, and newlines are still,
as documented, treated as spaces.
I don't remember the reasoning behind this but it's probably because
you have the ability in the format to say exactly what you're looking
for, so requiring newlines to match is less error-prone. Whatever
the reason, it's documented so it should work.
Fixes #10862.
Change-Id: I7aca93470f9608429c9f01ea23cbdd1f17410ae5
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
Aamir Khan (Gerrit)
Jun 11, 2015, 10:23:04 AM6/11/15
to Rob Pike, Gobot Gobot, golang-co...@googlegroups.com
Aamir Khan has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 4:
(2 comments)
https://go-review.googlesource.com/#/c/10779/4/src/fmt/scan.go
File src/fmt/scan.go:
Line 38: // unless a format is specified, in which case a newline is
treated as EOF.
I think this is incorrect, newline is newline if format is specified and
must match. Is EOF for Scanln, Fscanln, Sscanln
Line 44: // specified, in which case a newline is treated as EOF.
same as above
--
https://go-review.googlesource.com/10779
Gerrit-Reviewer: Aamir Khan <syst3...@gmail.com>
Gerrit-Reviewer: Gobot Gobot <go...@golang.org>
Gerrit-Reviewer: Rob Pike <r...@golang.org>
Rob Pike (Gerrit)
Jun 11, 2015, 4:43:26 PM6/11/15
to Rob Pike, Gobot Gobot, Aamir Khan, golang-co...@googlegroups.com
Rob Pike has abandoned this change.
Change subject: fmt: require newlines to match when scanning with a format
......................................................................
Abandoned
Rethinking this.
Change subject: fmt: require newlines to match when scanning with a format
......................................................................
Abandoned
Rethinking this.
Rob Pike (Gerrit)
Jun 12, 2015, 3:57:03 PM6/12/15
to Ian Lance Taylor, Rob Pike, golang-co...@googlegroups.com
Rob Pike uploaded a change:
https://go-review.googlesource.com/10991
fmt: require newlines to match when scanning with a format
Scan etc.: newlines are spaces.
Scanln etc.: newlines terminate the scan.
Scanf etc.: newlines must match in input and format.
The code did not implement this behavior in all cases,
especially for Scanf. Make it behave:
- Fix the handling of spaces and newlines in ss.Advance.
The code is longer but now behaves as it should.
- Delete the reuse of the current ss in newScanState.
There is really no need, since it's only used in recursive
calls to Scan etc., and the flags are likely wrong. Simpler
just to allocate a new one every time, and likelier to
be correct.
Fixes #10862.
Change-Id: If060ac021017346723b0d62de4e5a305da898f68
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 149 insertions(+), 42 deletions(-)
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
diff --git a/src/fmt/doc.go b/src/fmt/doc.go
index cbca6ab..a5fb513 100644
--- a/src/fmt/doc.go
+++ b/src/fmt/doc.go
@@ -237,11 +237,24 @@
An analogous set of functions scans formatted text to yield
values. Scan, Scanf and Scanln read from os.Stdin; Fscan,
Fscanf and Fscanln read from a specified io.Reader; Sscan,
- Sscanf and Sscanln read from an argument string. Scanln,
- Fscanln and Sscanln stop scanning at a newline and require that
- the items be followed by one; Scanf, Fscanf and Sscanf require
- newlines in the input to match newlines in the format; the other
- routines treat newlines as spaces.
+ Sscanf and Sscanln read from an argument string.
- newlines in the input to match newlines in the format; the other
- routines treat newlines as spaces.
+
+ Scan, Fscan, Sscan treat newlines in the input as spaces.
+
+ Scanln, Fscanln and Sscanln stop scanning at a newline and
+ require that the items be followed by a newline or EOF.
+
+ Scanf, Fscanf and Sscanf require that (after skipping spaces)
+ newlines in the format are matched by newlines in the input
+ Scanf, Fscanf and Sscanf require that (after skipping spaces)
+ and vice versa. This behavior differs from the corresponding
+ routines in C, which uniformly treat newlines as spaces.
+
+ When scanning with Scanf, Fscanf, and Sscanf, all non-empty
+ runs of space characters (except newline) are equivalent
+ to a single space in both the format and the input. With
+ that proviso, text in the format string must match the input
+ text; scanning stops if it does not, with the return value
+ of the function indicating the number of arguments scanned.
Scanf, Fscanf, and Sscanf parse the arguments according to a
format string, analogous to that of Printf. For example, %x
five runes of input will be read to scan a string) but there
is no syntax for scanning with a precision (no %5.2f, just
%5f).
-
- When scanning with a format, all non-empty runs of space
- characters (except newline) are equivalent to a single
- space in both the format and the input. With that proviso,
- text in the format string must match the input text; scanning
- stops if it does not, with the return value of the function
- indicating the number of arguments scanned.
In all the scanning functions, a carriage return followed
immediately by a newline is treated as a plain newline
diff --git a/src/fmt/scan.go b/src/fmt/scan.go
index 9572530..d6b9b79 100644
--- a/src/fmt/scan.go
+++ b/src/fmt/scan.go
@@ -34,16 +34,16 @@
ReadRune() (r rune, size int, err error)
// UnreadRune causes the next call to ReadRune to return the same rune.
UnreadRune() error
- // SkipSpace skips space in the input. Newlines are treated as space
- // unless the scan operation is Scanln, Fscanln or Sscanln, in which case
- // a newline is treated as EOF.
+ // SkipSpace skips space in the input. Newlines are treated appropriately
+ // for the operation being performed; see the package documentation
+ // for more information.
SkipSpace()
// Token skips space in the input if skipSpace is true, then returns the
// run of Unicode code points c satisfying f(c). If f is nil,
// !unicode.IsSpace(c) is used; that is, the token will hold non-space
- // characters. Newlines are treated as space unless the scan operation
- // is Scanln, Fscanln or Sscanln, in which case a newline is treated as
- // EOF. The returned slice points to shared data that may be overwritten
+ // characters. Newlines are treated appropriately for the operation being
+ // performed; see the package documentation for more information.
+ // The returned slice points to shared data that may be overwritten
// by the next call to Token, a call to a Scan function using the
ScanState
// as input, or when the calling Scan method returns.
Token(skipSpace bool, f func(rune) bool) (token []byte, err error)
@@ -82,6 +82,7 @@
// space-separated values into successive arguments as determined by
// the format. It returns the number of items successfully scanned.
// If that is less than the number of arguments, err will report why.
+// Newlines in the input must match newlines in the format.
func Scanf(format string, a ...interface{}) (n int, err error) {
return Fscanf(os.Stdin, format, a...)
}
@@ -114,6 +115,7 @@
// Sscanf scans the argument string, storing successive space-separated
// values into successive arguments as determined by the format. It
// returns the number of items successfully parsed.
+// Newlines in the input must match newlines in the format.
func Sscanf(str string, format string, a ...interface{}) (n int, err
error) {
return Fscanf((*stringReader)(&str), format, a...)
}
@@ -141,6 +143,7 @@
// Fscanf scans text read from r, storing successive space-separated
// values into successive arguments as determined by the format. It
// returns the number of items successfully parsed.
+// Newlines in the input must match newlines in the format.
func Fscanf(r io.Reader, format string, a ...interface{}) (n int, err
error) {
s, old := newScanState(r, false, false)
n, err = s.doScanf(format, a)
@@ -388,17 +391,6 @@
// newScanState allocates a new ss struct or grab a cached one.
func newScanState(r io.Reader, nlIsSpace, nlIsEnd bool) (s *ss, old ssave)
{
- // If the reader is a *ss, then we've got a recursive
- // call to Scan, so re-use the scan state.
- s, ok := r.(*ss)
- if ok {
- old = s.ssave
- s.limit = s.argLimit
- s.nlIsEnd = nlIsEnd || s.nlIsEnd
- s.nlIsSpace = nlIsSpace
- return
- }
-
s = ssFree.Get().(*ss)
if rr, ok := r.(io.RuneReader); ok {
s.rr = rr
@@ -1057,8 +1049,8 @@
s.scanOne('v', arg)
numProcessed++
}
- // Check for newline if required.
- if !s.nlIsSpace {
+ // Check for newline (or EOF) if required (Scanln etc.).
+ if s.nlIsEnd {
for {
r := s.getRune()
if r == '\n' || r == eof {
@@ -1074,12 +1066,13 @@
}
// advance determines whether the next characters in the input match
-// those of the format. It returns the number of bytes (sic) consumed
-// in the format. Newlines included, all runs of space characters in
-// either input or format behave as a single space. This routine also
-// handles the %% case. If the return value is zero, either format
-// starts with a % (with no following %) or the input is empty.
-// If it is negative, the input did not match the string.
+// those of the format. It returns the number of bytes (sic) consumed
+// in the format. All runs of space characters in either input or
+// format behave as a single space. Newlines are special, though:
+// newlines in the format must match those in the input and vice versa.
// advance determines whether the next characters in the input match
-// those of the format. It returns the number of bytes (sic) consumed
-// in the format. Newlines included, all runs of space characters in
-// either input or format behave as a single space. This routine also
-// handles the %% case. If the return value is zero, either format
-// starts with a % (with no following %) or the input is empty.
-// If it is negative, the input did not match the string.
+// those of the format. It returns the number of bytes (sic) consumed
+// in the format. All runs of space characters in either input or
+// format behave as a single space. Newlines are special, though:
+// This routine also handles the %% case. If the return value is zero,
+// either format starts with a % (with no following %) or the input
+// is empty. If it is negative, the input did not match the string.
func (s *ss) advance(format string) (i int) {
for i < len(format) {
fmtc, w := utf8.DecodeRuneInString(format[i:])
@@ -1092,24 +1085,45 @@
for i < len(format) {
fmtc, w := utf8.DecodeRuneInString(format[i:])
i += w // skip the first %
}
sawSpace := false
+ wasNewline := false
+ // Skip spaces in format but absorb at most one newline.
}
sawSpace := false
+ wasNewline := false
for isSpace(fmtc) && i < len(format) {
+ if fmtc == '\n' {
+ if wasNewline { // Already saw one; stop here.
+ break
+ }
+ wasNewline = true
+ }
--- a/src/fmt/scan_test.go
+++ b/src/fmt/scan_test.go
@@ -1044,3 +1044,90 @@
t.Errorf("odd count: got count, err = %d, %v; expected 0, error", n, err)
}
}
+
}
}
+
+func TestScanNewlinesAreSpaces(t *testing.T) {
+ var a, b int
+ var tests = []struct {
+ var a, b int
+ name string
+ text string
+ count int
+ ok bool
+ }{
+ {"newlines", "1\n2\n", 2, true},
+ {"no final newline", "1\n2", 2, true},
+ {"newlines with spaces ", "1 \n 2 \n", 2, true},
+ {"no final newline with spaces", "1 \n 2", 2, true},
+ }
+ for _, test := range tests {
+ n, err := Sscan(test.text, &a, &b)
+ if n != test.count {
+ t.Errorf("%s: expected to scan %d item(s), scanned %d", test.name,
test.count, n)
+ }
+ if test.ok && err != nil {
+ t.Errorf("%s: unexpected error: %s", test.name, err)
+ }
+ if !test.ok && err == nil {
+ t.Errorf("%s: expected error; got none", test.name)
+ }
+ }
+}
+
+func TestScanlnNewlinesTerminate(t *testing.T) {
+ var a, b int
+ var tests = []struct {
+ name string
+ text string
+ count int
+ ok bool
+ }{
+ {"one line one item", "1\n", 1, false},
+ {"one line two items with spaces ", " 1 2 \n", 2, true},
+ {"one line two items no newline", " 1 2", 2, true},
+ {"two lines two items", "1\n2\n", 1, false},
+ }
+ for _, test := range tests {
+ Println(test.name)
+ n, err := Sscanln(test.text, &a, &b)
+ if n != test.count {
+ t.Errorf("%s: expected to scan %d item(s), scanned %d", test.name,
test.count, n)
+ }
+ if test.ok && err != nil {
+ t.Errorf("%s: unexpected error: %s", test.name, err)
+ }
+ if !test.ok && err == nil {
+ t.Errorf("%s: expected error; got none", test.name)
+ }
+ }
+}
+
+func TestScanfNewlineMatchFormat(t *testing.T) {
+ var a, b int
+ var tests = []struct {
+ name string
+ text string
+ format string
+ count int
+ ok bool
+ }{
+ {"newline in both", "1\n2", "%d\n%d\n", 2, true},
+ {"newline in input", "1\n2", "%d %d", 1, false},
+ {"space-newline in input", "1 \n2", "%d %d", 1, false},
+ {"newline in format", "1 2", "%d\n%d", 1, false},
+ {"space-newline in format", "1 2", "%d \n%d", 1, false},
+ {"space-newline in both", "1 \n2", "%d \n%d", 2, true},
+ {"extra space in format", "1\n2", "%d\n %d", 2, true},
+ {"two extra spaces in format", "1\n2", "%d \n %d", 2, true},
+ }
+ for _, test := range tests {
+ n, err := Sscanf(test.text, test.format, &a, &b)
+ if n != test.count {
+ t.Errorf("%s: expected to scan %d item(s), scanned %d", test.name,
test.count, n)
+ }
+ if test.ok && err != nil {
+ t.Errorf("%s: unexpected error: %s", test.name, err)
+ }
+ if !test.ok && err == nil {
+ t.Errorf("%s: expected error; got none", test.name)
+ }
+ }
+}
--
https://go-review.googlesource.com/10991
Andrew Gerrand (Gerrit)
Jun 12, 2015, 4:05:00 PM6/12/15
to Rob Pike, golang-co...@googlegroups.com
Andrew Gerrand has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1: Code-Review+2
(2 comments)
https://go-review.googlesource.com/#/c/10991/1/src/fmt/scan_test.go
File src/fmt/scan_test.go:
Line 1054: ok bool
seems unnecessary; there are no expected failure cases
Line 1089: Println(test.name)
debugging dreg?
--
https://go-review.googlesource.com/10991
Gerrit-Reviewer: Andrew Gerrand <a...@golang.org>
Gerrit-HasComments: Yes
fmt: require newlines to match when scanning with a format
(2 comments)
https://go-review.googlesource.com/#/c/10991/1/src/fmt/scan_test.go
File src/fmt/scan_test.go:
Line 1054: ok bool
seems unnecessary; there are no expected failure cases
Line 1089: Println(test.name)
debugging dreg?
--
https://go-review.googlesource.com/10991
Gerrit-Reviewer: Andrew Gerrand <a...@golang.org>
Gerrit-HasComments: Yes
Rob Pike (Gerrit)
Jun 12, 2015, 4:13:43 PM6/12/15
to Rob Pike, Andrew Gerrand, golang-co...@googlegroups.com
Reviewers: Andrew Gerrand
Rob Pike uploaded a new patch set:
Rob Pike uploaded a new patch set:
https://go-review.googlesource.com/10991
fmt: require newlines to match when scanning with a format
The documentation says that newlines behave like this:
Scan etc.: newlines are spaces.
Scanln etc.: newlines terminate the scan.
Scanf etc.: newlines must match in input and format.
The code did not implement this behavior in all cases,
especially for Scanf. Make it behave:
- Fix the handling of spaces and newlines in ss.Advance.
The code is longer but now behaves as it should.
- Delete the reuse of the current ss in newScanState.
There is really no need, since it's only used in recursive
calls to Scan etc., and the flags are likely wrong. Simpler
just to allocate a new one every time, and likelier to
be correct.
Fixes #10862.
Change-Id: If060ac021017346723b0d62de4e5a305da898f68
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 144 insertions(+), 42 deletions(-)
fmt: require newlines to match when scanning with a format
The documentation says that newlines behave like this:
Scan etc.: newlines are spaces.
Scanln etc.: newlines terminate the scan.
Scanf etc.: newlines must match in input and format.
The code did not implement this behavior in all cases,
especially for Scanf. Make it behave:
- Fix the handling of spaces and newlines in ss.Advance.
The code is longer but now behaves as it should.
- Delete the reuse of the current ss in newScanState.
There is really no need, since it's only used in recursive
calls to Scan etc., and the flags are likely wrong. Simpler
just to allocate a new one every time, and likelier to
be correct.
Fixes #10862.
Change-Id: If060ac021017346723b0d62de4e5a305da898f68
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
Rob Pike (Gerrit)
Jun 12, 2015, 4:14:10 PM6/12/15
to Rob Pike, Andrew Gerrand, golang-co...@googlegroups.com
Rob Pike has posted comments on this change.
fmt: require newlines to match when scanning with a format
Patch Set 1:
(2 comments)
https://go-review.googlesource.com/#/c/10991/1/src/fmt/scan_test.go
File src/fmt/scan_test.go:
Line 1054: ok bool
> seems unnecessary; there are no expected failure cases
Done
Done
fmt: require newlines to match when scanning with a format
(2 comments)
https://go-review.googlesource.com/#/c/10991/1/src/fmt/scan_test.go
File src/fmt/scan_test.go:
Line 1054: ok bool
> seems unnecessary; there are no expected failure cases
Done
Rob Pike (Gerrit)
Jun 12, 2015, 4:14:18 PM6/12/15
to Rob Pike, golang-...@googlegroups.com, Andrew Gerrand, golang-co...@googlegroups.com
Rob Pike has submitted this change and it was merged.
fmt: require newlines to match when scanning with a format
The documentation says that newlines behave like this:
Scan etc.: newlines are spaces.
Scanln etc.: newlines terminate the scan.
Scanf etc.: newlines must match in input and format.
The code did not implement this behavior in all cases,
especially for Scanf. Make it behave:
- Fix the handling of spaces and newlines in ss.Advance.
The code is longer but now behaves as it should.
- Delete the reuse of the current ss in newScanState.
There is really no need, since it's only used in recursive
calls to Scan etc., and the flags are likely wrong. Simpler
just to allocate a new one every time, and likelier to
be correct.
Fixes #10862.
Change-Id: If060ac021017346723b0d62de4e5a305da898f68
Reviewed-on: https://go-review.googlesource.com/10991
Reviewed-by: Andrew Gerrand <a...@golang.org>
Approvals:
Andrew Gerrand: Looks good to me, approved
fmt: require newlines to match when scanning with a format
The documentation says that newlines behave like this:
Scan etc.: newlines are spaces.
Scanln etc.: newlines terminate the scan.
Scanf etc.: newlines must match in input and format.
The code did not implement this behavior in all cases,
especially for Scanf. Make it behave:
- Fix the handling of spaces and newlines in ss.Advance.
The code is longer but now behaves as it should.
- Delete the reuse of the current ss in newScanState.
There is really no need, since it's only used in recursive
calls to Scan etc., and the flags are likely wrong. Simpler
just to allocate a new one every time, and likelier to
be correct.
Fixes #10862.
Change-Id: If060ac021017346723b0d62de4e5a305da898f68
Reviewed-by: Andrew Gerrand <a...@golang.org>
---
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
3 files changed, 144 insertions(+), 42 deletions(-)
M src/fmt/doc.go
M src/fmt/scan.go
M src/fmt/scan_test.go
Approvals:
Andrew Gerrand: Looks good to me, approved
Reply all
Reply to author
Forward
0 new messages