PersistId

[Michael Snoyman]

Starting with Persistent 0.6, IDs in Persistent are significantly more
transparent, working with just a normal newtype with a phantom type
parameter[1]. This fact is actually irrelevant for the current
discussion, but may help people unfamiliar with the internals
understand what's going on.

In the past (version 0.4), an ID was actually an Int64 under the
surface, which worked really nicely for SQL. However, this didn't work
for MongoDB, which uses non-numeric keys. So we switched the
representation to PersistValue, our basic dynamic datatype in
persistent. And all was good.

Except that our SinglePiece instance is no longer very helpful!
Remember, SinglePiece[2] has a function "fromSinglePiece :: Text ->
Maybe s". Let's say we call `fromSinglePiece "abc"`. Should this be
Nothing, or `Just (PersistText "abc")`? Well, for SQL the former and
other backends, the latter. The problem is that there's nothing to
tell us about this at the type level.

The result is that any paths that use IDs will now accept any
arbitrary value. When SQL sees that we're trying to compare an INTEGER
field (the id) to a VARCHAR, it barfs, resulting in 500s.

The question is: what do we do? Add an extra parameter to the
fromSinglePiece function for an environment so we can control how it
functions? Add special handling for IDs? Have SQL check if an ID is
non-numeric and return a 404 instead of a 500? I'm open to
suggestions.

Michael

[1] https://github.com/yesodweb/persistent/blob/master/persistent/Database/Persist/Base.hs#L289
[2] http://hackage.haskell.org/packages/archive/path-pieces/0.0.0/doc/html/Web-PathPieces.html

[Michael Snoyman]

OK, I just put up a new branch[1] that uses the first approach I
mentioned (the extra parameter). Seems to be working OK. Anyone want
to review/provide feedback?

[1] https://github.com/yesodweb/yesod/tree/multiparam-singlepiece/yesod/scaffold

[Greg Weber]

I am wondering how you are going to determine what the backend is for that TODO. We have to consider the usage of multiple backends. When using a non-sql backend it is often not the only backend (due to legacy concerns or otherwise just playing to the strengths of different backends).

Fundamentally our Key type is too vague. It is for any PersistValue, but we only want a PersistInt64 or a PersistObjectId (PersistByteString). Is it possible to make the Key type backend specific? Or can we tie the Id alias to a specific backend?

[Michael Snoyman]

Good thinking (at least I hope it is)... that's exactly the conclusion
I came to today. Let me spell it out a bit more. The theoretical best
approach is to have an associated data type as part of
PersistentBackend, something like:

class PersistBackend m where
data Key m entity

This works great in theory... until you get to foreign keys. Take this example:

Address
street Text
city Text
Person
name Text
address AddressId

Currently, this gets translated into roughly:

data Address = Address { street :: Text, city :: Text}
data Person = Person { name :: Text, address :: Key Address }

However, under this new system, the value for the address record is
insufficient, instead we would need something like "Key SqlPersist
Address". But doing so would mean that our entity only works for SQL
(or alternatively, only for Mongo). So instead, we'd want to do
something like:

data Person backend = Person { name :: Text, address :: Key backend Address }

I've considered this in the past, but dismissed it as infeasible,
since no one wants to type "Person SqlPersist" everywhere. But after
much thought today, I realized this neatly solves far too many issues
(SinglePiece instance, accidently using a Mongo ID in SQL, etc) for it
to be ignored. Then I came up with a nice little approach to make this
more palatable:

data PersonG backend = Person { name :: Text, address :: Key backend Address }
type Person = PersonG SqlPersist
type PersonId = Key SqlPersist Person

Which PersistBackend we use would be an argument to the Template
Haskell code. (This argument would contain some other settings, such
as naming convention for the tables and columns. At this point, we can
go back to our original definition of the SinglePiece class, and have
instances along the lines of:

data instance Key SqlPersist entity = SqlKey { unSqlKey :: Int64 }
data instance Key MongoPersist entity = MongoKey { unMongoKey :: ByteString }
instance SinglePiece (Key SqlPersist entity) where
toSinglePiece (SqlKey i) = pack $ show i
fromSinglePiece t = SqlKey $ read $ unpack t -- obviously
something more robust...

The great thing about this approach is it should have very minimal
impact on existing code, just needing to modify the call to mkPersist
to past the correct settings. I feel much more confident about this
proposal than the previous one.

Any thoughts?

Michael