Account Options

  1. Sign in
The old Google Groups will be going away soon, but your browser is incompatible with the new version.
Google Groups Home
« Groups Home
Added attoparsec unit test for unicode parsing
There are currently too many topics in this group that display first. To make this topic appear first, remove this option from another topic.
There was an error processing your request. Please try again.
flag
  Messages 26 - 33 of 33 - Collapse all  -  Translate all to Translated (View all originals) < Older 
The group you are posting to is a Usenet group. Messages posted to this group will make your email address visible to anyone on the Internet.
Your reply message has not been sent.
Your post was successful
 
From:
To:
Cc:
Followup To:
Add Cc | Add Followup-to | Edit Subject
Subject:
Validation:
For verification purposes please type the characters you see in the picture below or the numbers you hear by clicking the accessibility icon. Listen and type the numbers you hear
 
Michele Tartara  
View profile  
 More options Nov 13 2012, 4:46 am
From: Michele Tartara <mtart...@google.com>
Date: Tue, 13 Nov 2012 10:46:35 +0100
Local: Tues, Nov 13 2012 4:46 am
Subject: Re: [PATCH master 2/2] Added attoparsec unit test for unicode parsing

List added back

On Tue, Nov 13, 2012 at 10:46 AM, Michele Tartara <mtart...@google.com>wrote:


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Iustin Pop  
View profile  
 More options Nov 13 2012, 4:52 am
From: Iustin Pop <ius...@google.com>
Date: Tue, 13 Nov 2012 10:52:55 +0100
Local: Tues, Nov 13 2012 4:52 am
Subject: Re: [PATCH master 2/2] Added attoparsec unit test for unicode parsing

On Tue, Nov 13, 2012 at 10:46:35AM +0100, Michele Tartara wrote:
> List added back

> On Tue, Nov 13, 2012 at 10:46 AM, Michele Tartara <mtart...@google.com>wrote:

> > Oh, no, as you can see from the patch itself, CPP is not used anymore.

> > Should I send a fixed patch with the modified comment, or can you fix it
> > while pushing it?

I'll fix while pushing. Just asking for confirmation :)

LGTM & thanks,
iustin


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Iustin Pop  
View profile  
 More options Nov 13 2012, 5:00 am
From: Iustin Pop <ius...@google.com>
Date: Tue, 13 Nov 2012 11:00:55 +0100
Local: Tues, Nov 13 2012 5:00 am
Subject: Re: [PATCH master 2/2] Added attoparsec unit test for unicode parsing

On Tue, Nov 13, 2012 at 10:52:55AM +0100, Iustin Pop wrote:
> On Tue, Nov 13, 2012 at 10:46:35AM +0100, Michele Tartara wrote:
> > List added back

> > On Tue, Nov 13, 2012 at 10:46 AM, Michele Tartara <mtart...@google.com>wrote:

> > > Oh, no, as you can see from the patch itself, CPP is not used anymore.

> > > Should I send a fixed patch with the modified comment, or can you fix it
> > > while pushing it?

> I'll fix while pushing. Just asking for confirmation :)

> LGTM & thanks,

Actually I was too careless, there are a few code style issues, and one
actual code bug in your test code.

Style issues:

- trailing whitespace; please configure your editor appropriately
- missing docstrings; we do want docstrings even on constants (e.g.
  "Unicode test string, first part.", "Unicode test string, second
  part.")

And to the code bug: never, ever "let (Right foo) = …". This will lead
to a runtime error, which is a big no-no. You should always deconstruct
via case:

  case A.parseOnly simpleParser … of
    Right y -> …
    Left msg -> failTest $ "Failed to parse: " ++ msg

As a side-comment, defining a separate type (SplitString) just for one
use seems over doing it. I personally would just return a tuple (String,
String) from simpleParser…

thanks,
iustin


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Michele Tartara  
View profile  
 More options Nov 13 2012, 5:11 am
From: Michele Tartara <mtart...@google.com>
Date: Tue, 13 Nov 2012 11:11:17 +0100
Local: Tues, Nov 13 2012 5:11 am
Subject: Re: [PATCH master 2/2] Added attoparsec unit test for unicode parsing

Ok, I'll fix this immediately.

Thanks,
Michele


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Iustin Pop  
View profile  
 More options Nov 13 2012, 5:28 am
From: Iustin Pop <ius...@google.com>
Date: Tue, 13 Nov 2012 11:28:41 +0100
Local: Tues, Nov 13 2012 5:28 am
Subject: Re: [PATCH master 2/2] Added attoparsec unit test for unicode parsing

FYI, this would be detected and enforced by the compiler, but we're
still supporting ghc 6.12, whereas this flag appeared only in 7.2. Me
wants newer compilers!!

Hmmm, I wonder if I can't change the build system to detect this and
enable this (not-automatically enable-able via -Wall) flag…

thanks,
iustin


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Discussion subject changed to "Added attoparsec unit test for Unicode parsing" by Michele Tartara
Michele Tartara  
View profile  
 More options Nov 13 2012, 6:31 am
From: Michele Tartara <mtart...@google.com>
Date: Tue, 13 Nov 2012 12:31:13 +0100
Local: Tues, Nov 13 2012 6:31 am
Subject: [PATCH master] Added attoparsec unit test for Unicode parsing
Attoparsec is known to have had issues with parsing non-ASCII strings.
This test makes sure that parsing of Unicode characters works fine.

Signed-off-by: Michele Tartara <mtart...@google.com>
---
 Makefile.am                     |    1 +
 htest/Test/Ganeti/Attoparsec.hs |   69 +++++++++++++++++++++++++++++++++++++++
 htest/test.hs                   |    2 +
 3 files changed, 72 insertions(+), 0 deletions(-)
 create mode 100644 htest/Test/Ganeti/Attoparsec.hs

diff --git a/Makefile.am b/Makefile.am
index ea38407..d6a0e28 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -473,6 +473,7 @@ HS_LIB_SRCS = \
        htools/Ganeti/Utils.hs

 HS_TEST_SRCS = \
+       htest/Test/Ganeti/Attoparsec.hs \
        htest/Test/Ganeti/BasicTypes.hs \
        htest/Test/Ganeti/Common.hs \
        htest/Test/Ganeti/Confd/Utils.hs \
diff --git a/htest/Test/Ganeti/Attoparsec.hs b/htest/Test/Ganeti/Attoparsec.hs
new file mode 100644
index 0000000..14ddd9c
--- /dev/null
+++ b/htest/Test/Ganeti/Attoparsec.hs
@@ -0,0 +1,69 @@
+{-# LANGUAGE TemplateHaskell #-}
+
+{-| Unittests for Attoparsec support for unicode -}
+
+{-
+
+Copyright (C) 2009, 2010, 2011, 2012 Google Inc.
+
+This program is free software; you can redistribute it and/or modify
+it under the terms of the GNU General Public License as published by
+the Free Software Foundation; either version 2 of the License, or
+(at your option) any later version.
+
+This program is distributed in the hope that it will be useful, but
+WITHOUT ANY WARRANTY; without even the implied warranty of
+MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+General Public License for more details.
+
+You should have received a copy of the GNU General Public License
+along with this program; if not, write to the Free Software
+Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA
+02110-1301, USA.
+
+-}
+
+module Test.Ganeti.Attoparsec (testAttoparsec) where
+
+
+import Test.QuickCheck
+
+import Test.Ganeti.TestHelper
+import Test.Ganeti.TestCommon
+
+import qualified Data.Attoparsec.Text as A
+import Data.Attoparsec.Text (Parser)
+import Data.Text (pack, unpack)
+
+
+{-| Unicode test string, first part -}
+part1 :: String
+part1 = "äßĉ"
+
+{-| Unicode test string, second part -}
+part2 :: String
+part2 = "ščق"
+
+
+{-| Simple parser able to split a string in two parts, name and value,
+    separated by a '=' sign -}
+simpleParser :: Parser (String, String)
+simpleParser = do
+  n <- A.takeTill (\c -> A.isHorizontalSpace c || c == '=')
+  A.skipWhile A.isHorizontalSpace
+  _ <- A.char '='
+  A.skipWhile A.isHorizontalSpace
+  v <- A.takeTill A.isEndOfLine
+  return (unpack n, unpack v)
+
+
+-- | Tests whether a Unicode string is still Unicode after being parsed
+prop_parserSupportsUnicode :: Property
+prop_parserSupportsUnicode = case A.parseOnly simpleParser text of
+    Right (name, value) -> (name ==? part1) .&&. (value ==? part2)
+    Left msg -> failTest $ "Failed to parse: " ++ msg
+  where text = Data.Text.pack $ part1 ++ "  = \t" ++ part2
+
+testSuite "Attoparsec"
+          [ 'prop_parserSupportsUnicode
+          ]
diff --git a/htest/test.hs b/htest/test.hs
index 5180878..b41fb99 100644
--- a/htest/test.hs
+++ b/htest/test.hs
@@ -30,6 +30,7 @@ import Test.Framework
 import System.Environment (getArgs)

 import Test.Ganeti.TestImports ()
+import Test.Ganeti.Attoparsec
 import Test.Ganeti.BasicTypes
 import Test.Ganeti.Common
 import Test.Ganeti.Confd.Utils
@@ -74,6 +75,7 @@ defOpts = TestOptions
 allTests :: [Test]
 allTests =
   [ testBasicTypes
+  , testAttoparsec
   , testCommon
   , testConfd_Utils
   , testDaemon
--
1.7.7.3


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Iustin Pop  
View profile  
 More options Nov 13 2012, 6:39 am
From: Iustin Pop <ius...@google.com>
Date: Tue, 13 Nov 2012 12:39:54 +0100
Local: Tues, Nov 13 2012 6:39 am
Subject: Re: [PATCH master] Added attoparsec unit test for Unicode parsing

On Tue, Nov 13, 2012 at 12:31:13PM +0100, Michele Tartara wrote:
> Attoparsec is known to have had issues with parsing non-ASCII strings.
> This test makes sure that parsing of Unicode characters works fine.

LGTM, thanks. I'll push with just some trivial docstring/style changes.

iustin


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
Michele Tartara  
View profile  
 More options Nov 13 2012, 6:40 am
From: Michele Tartara <mtart...@google.com>
Date: Tue, 13 Nov 2012 12:40:37 +0100
Local: Tues, Nov 13 2012 6:40 am
Subject: Re: [PATCH master] Added attoparsec unit test for Unicode parsing

Ok, thanks, Next time I'll use the right docstring format.

Michele


 
You must Sign in before you can post messages.
To post a message you must first join this group.
Please update your nickname on the subscription settings page before posting.
You do not have the permission required to post.
End of messages < Older 
« Back to Discussions « Newer topic     Older topic »