Better developper experience?

[Chrissom]

Hi,

I use ReactiveMongo through several projects. RM is clearly very good but could be better from a dev UX perspective (productivity and readability). For example in the last version (0.12) I found some UX issues:

- productivity :

RM provides a MongoController trait to write queries faster. Awesome :) Now if I have 2 collections, I have to write some identical queries (CRUD). Only collection attributes change. Imagine if I have to write 4 or 5 controllers... a DAO approach could help and avoid me to write and write the same things (sadly ReactiveMongo-Extensions is not maintained). 

I use Play to write web app faster and RM forces me to write a lot of code. From my perspective, RM has a productivity issue.

- readability (collection): 

using map or flatMap is sometimes annoying and add effort to read the code (and in Scala it's really really easy to write code hard to read... but that's another problem). Here is some example:

class UserDao @Inject()(mongodb: ReactiveMongoApi) {

  def collection = mongodb.database.map(_[JSONCollection]("users"))
}

//
// WHY NOT (ALSO) ?
//

class UserDao @Inject()() extends JsonMongoCollection("users") {

  // a JSONCollection is available here. 
  // It's more simple, easy to read and productive. 

  // No more '...map(_[...]("...")])' to read.
  // 99% of time, all I want is an access to my collection. Nothing else.
}

Providing a trait is easy to read and more productive.

- readability (query): 

same reason above. map and flatMap...

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection.flatMap { _
  .find(selector)
  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))

  .cursor[User](ReadPreference.Primary)
  .collect[List](Int.MaxValue, Cursor.FailOnError[List[User]]())
}

//
// WHY NOT (ALSO) ?
// - no flatMap
// - no .options
// - no QueryOpts (no import needed...)
// - a ctrl-spc inside an IDE list all options available to the developer (no need to search in the RM docs)
// 

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection
  .find(selector)
  .sort(sort)
  .skip((page - 1) * pageSize)

  .limit(pageSize)

  .cursor[User]()
  .collect[List]()

chaining .find .sort .skip .limit is more easy and confortable to read.

- readability (json / bson DSL): 

a DSL could make the query more natural and easy to read (ReactiveMongo-Extensions provided an awesome json/bson DSL).

def findInText(text: String) = find(Json.obj(
    f"$$text" -> Json.obj(
      f"$$search" -> text)))

//
// WHY NOT (ALSO)?
//

def findInText(terms: String) = collection.find($text(terms))

- readability (criteria DSL): 

No need to talk more about that, there is an awesome project that provides a criteria DSL.

Conclusion

RM is awesome but it's a driver which provides core features at low level. With helpers. I get it. 

But when I create a new Scala+Mongo project I ask myself: is there a new library that provides the 3 golden features (top performance, high productivity and awesome readability)? RM is on top of the list but has a big lack on productivity and readability.

So, does the core RM developers want to make only a driver OR the most awesome Scala Mongo library for developers?

[Chrissom]

To go further, here is how I see the RM projects:

• reactivemongo-driver -- core features at low level (performance)
• reactivemongo-orm -- helper features at high level (productivity) -- DAO...
  
• reactivemongo-dsl -- bson / json DSL (readibility) -- bson, play-json, circe...
  
• reactivemongo-criteria -- criteria DSL (readibility)

With this modular approach, in the build.sbt you import only what you need.

[Cédric Chantepie]

Hi,

On Thursday, 11 May 2017 18:13:46 UTC+2, Chrissom wrote:

- productivity :

RM provides a MongoController trait to write queries faster. Awesome :) Now if I have 2 collections, I have to write some identical queries (CRUD).

2 collections ?

 

Only collection attributes change. Imagine if I have to write 4 or 5 controllers... a DAO approach could help and avoid me to write and write the same things (sadly ReactiveMongo-Extensions is not maintained). 

I use Play to write web app faster and RM forces me to write a lot of code. From my perspective, RM has a productivity issue.

Quite unclear ... 

- readability (collection): 

using map or flatMap is sometimes annoying and add effort to read the code (and in Scala it's really really easy to write code hard to read... but that's another problem). Here is some example:

class UserDao @Inject()(mongodb: ReactiveMongoApi) {

  def collection = mongodb.database.map(_[JSONCollection]("users"))
}

//
// WHY NOT (ALSO) ?
//

class UserDao @Inject()() extends JsonMongoCollection("users") {

  // a JSONCollection is available here. 
  // It's more simple, easy to read and productive. 

  // No more '...map(_[...]("...")])' to read.
  // 99% of time, all I want is an access to my collection. Nothing else.
}

Providing a trait is easy to read and more productive.

What would be provided by such trait ?

 

- readability (query): 

same reason above. map and flatMap...

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection.flatMap { _
  .find(selector)
  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))

  .cursor[User](ReadPreference.Primary)
  .collect[List](Int.MaxValue, Cursor.FailOnError[List[User]]())
}

//
// WHY NOT (ALSO) ?
// - no flatMap

That's async monadic reference. It could be replaced by a builder/factory, but I doubt it would bring improvement (and won't follow monadic rules).

 

// - no .options

?

 

// - no QueryOpts (no import needed...)

?

 

// - a ctrl-spc inside an IDE list all options available to the developer (no need to search in the RM docs)

?

// 

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection
  .find(selector)
  .sort(sort)
  .skip((page - 1) * pageSize)

  .limit(pageSize)

  .cursor[User]()
  .collect[List]()

chaining .find .sort .skip .limit is more easy and confortable to read.

QueryOpts.{skip,batchSize,flags,tailable,...} are provided. Gathering everything (mixing option, cursor resolution, sort clause) at the same QueryBuilder level doesn't sound great for me.

- readability (json / bson DSL): 

a DSL could make the query more natural and easy to read (ReactiveMongo-Extensions provided an awesome json/bson DSL).

def findInText(text: String) = find(Json.obj(
    f"$$text" -> Json.obj(
      f"$$search" -> text)))

//
// WHY NOT (ALSO)?
//

def findInText(terms: String) = collection.find($text(terms))

As already discussed in this MailingList, ReactiveMongo-extensions is no longer maintained.

Tier project project are welcome/free to provide such DSL, having the core dev maintaining one is not a priority.

 

- readability (criteria DSL): 

No need to talk more about that, there is an awesome project that provides a criteria DSL.

Conclusion

RM is awesome but it's a driver which provides core features at low level. With helpers. I get it. 

But when I create a new Scala+Mongo project I ask myself: is there a new library that provides the 3 golden features (top performance, high productivity and awesome readability)? RM is on top of the list but has a big lack on productivity and readability.

So, does the core RM developers want to make only a driver OR the most awesome Scala Mongo library for developers?

There is already a lot to do for the core dev, so yes the driver and the core lib (BSON/JSON support) are the priorities.

Developing the ecosystem with tier projects is welcome.

[Chrissom]

Le jeudi 11 mai 2017 19:30:50 UTC+2, Cédric Chantepie a écrit :

Hi,

On Thursday, 11 May 2017 18:13:46 UTC+2, Chrissom wrote:

- productivity :

RM provides a MongoController trait to write queries faster. Awesome :) Now if I have 2 collections, I have to write some identical queries (CRUD).

2 collections ?

2 MongoDB collection. For example, with Play, if I have: 

- UserController extends Controller with MongoController --> use the mongodb collection "users"

- PermissionController extends Controller with MongoController --> use the mongodb collection "permissions"

Both UserController and PermissionController could implement the find method with a sort/pagination option.

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection.flatMap { _
  .find(selector)
  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))
  .cursor[User](ReadPreference.Primary)
  .collect[List](Int.MaxValue, Cursor.FailOnError[List[User]]())
}

 

Only collection attributes change. Imagine if I have to write 4 or 5 controllers... a DAO approach could help and avoid me to write and write the same things (sadly ReactiveMongo-Extensions is not maintained). 

I use Play to write web app faster and RM forces me to write a lot of code. From my perspective, RM has a productivity issue.

Quite unclear ... 

See a more clear version with an example above (UserController, PermissionController and add 3 others controllers in your mind). At the end, I get 5x the find method with sort/pagination option.

 

- readability (collection): 

using map or flatMap is sometimes annoying and add effort to read the code (and in Scala it's really really easy to write code hard to read... but that's another problem). Here is some example:

class UserDao @Inject()(mongodb: ReactiveMongoApi) {

  def collection = mongodb.database.map(_[JSONCollection]("users"))
}

//
// WHY NOT (ALSO) ?
//

class UserDao @Inject()() extends JsonMongoCollection("users") {

  // a JSONCollection is available here. 
  // It's more simple, easy to read and productive. 

  // No more '...map(_[...]("...")])' to read.
  // 99% of time, all I want is an access to my collection. Nothing else.
}

Providing a trait is easy to read and more productive.

What would be provided by such trait ?

just a collection: Future[JSONCollection]. This one is free. This makes the code more easy to read, nothing else: readability experience. 

 

 

- readability (query): 

same reason above. map and flatMap...

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection.flatMap { _
  .find(selector)
  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))

  .cursor[User](ReadPreference.Primary)
  .collect[List](Int.MaxValue, Cursor.FailOnError[List[User]]())
}

//
// WHY NOT (ALSO) ?
// - no flatMap

That's async monadic reference. It could be replaced by a builder/factory, but I doubt it would bring improvement (and won't follow monadic rules).

 

// - no .options

?

no need to bring all options inside a method .options:

  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))
  .cursor[User](ReadPreference.Primary)

 

 

// - no QueryOpts (no import needed...)

?

Readability experience. QueryOpts makes reading the code boring. Why not?

  .sort(sort)
  .options(
    skip = 100,
    limit = 100))
  .cursor[User](ReadPreference.Primary)

 

 

// - a ctrl-spc inside an IDE list all options available to the developer (no need to search in the RM docs)

? 

// 

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection
  .find(selector)
  .sort(sort)
  .skip((page - 1) * pageSize)

  .limit(pageSize)

  .cursor[User]()
  .collect[List]()

chaining .find .sort .skip .limit is more easy and confortable to read.

QueryOpts.{skip,batchSize,flags,tailable,...} are provided. Gathering everything (mixing option, cursor resolution, sort clause) at the same QueryBuilder level doesn't sound great for me.

For the developer, it's easier. Inside my IDE I just have to use the shortcut "ctrl-space" to list all available methods. No need to switch to my brower and open the RM docs.

 

- readability (json / bson DSL): 

a DSL could make the query more natural and easy to read (ReactiveMongo-Extensions provided an awesome json/bson DSL).

def findInText(text: String) = find(Json.obj(
    f"$$text" -> Json.obj(
      f"$$search" -> text)))

//
// WHY NOT (ALSO)?
//

def findInText(terms: String) = collection.find($text(terms))

As already discussed in this MailingList, ReactiveMongo-extensions is no longer maintained.

Tier project project are welcome/free to provide such DSL, having the core dev maintaining one is not a priority.

a project "reactivemongo-dsl" could be hosted inside the RM organization? 

 

 

- readability (criteria DSL): 

No need to talk more about that, there is an awesome project that provides a criteria DSL.

Conclusion

RM is awesome but it's a driver which provides core features at low level. With helpers. I get it. 

But when I create a new Scala+Mongo project I ask myself: is there a new library that provides the 3 golden features (top performance, high productivity and awesome readability)? RM is on top of the list but has a big lack on productivity and readability.

So, does the core RM developers want to make only a driver OR the most awesome Scala Mongo library for developers?

There is already a lot to do for the core dev, so yes the driver and the core lib (BSON/JSON support) are the priorities.

Developing the ecosystem with tier projects is welcome.

Nice to hear that :) Last question. Why ReactiveMongo-Extensions which provide helpers to be productive (DAO, DSL, Criteria) is no longer maintained?

Thanks for your answers Cédric.

[Cédric Chantepie]

2 MongoDB collection. For example, with Play, if I have: 

- UserController extends Controller with MongoController --> use the mongodb collection "users"

- PermissionController extends Controller with MongoController --> use the mongodb collection "permissions"

Both UserController and PermissionController could implement the find method with a sort/pagination option.

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection.flatMap { _
  .find(selector)
  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))
  .cursor[User](ReadPreference.Primary)
  .collect[List](Int.MaxValue, Cursor.FailOnError[List[User]]())
}

If you mean having a trait focusing a controller on a collection, I can hardly see the benefit.

There the find is quite a duplicate of the QueryBuilder one.

If you mean having a pagination utility why not.

 

 

Only collection attributes change. Imagine if I have to write 4 or 5 controllers... a DAO approach could help and avoid me to write and write the same things (sadly ReactiveMongo-Extensions is not maintained). 

I use Play to write web app faster and RM forces me to write a lot of code. From my perspective, RM has a productivity issue.

Quite unclear ... 

See a more clear version with an example above (UserController, PermissionController and add 3 others controllers in your mind). At the end, I get 5x the find method with sort/pagination option.

See comment before

 

 

- readability (collection): 

using map or flatMap is sometimes annoying and add effort to read the code (and in Scala it's really really easy to write code hard to read... but that's another problem). Here is some example:

class UserDao @Inject()(mongodb: ReactiveMongoApi) {

  def collection = mongodb.database.map(_[JSONCollection]("users"))
}

//
// WHY NOT (ALSO) ?
//

class UserDao @Inject()() extends JsonMongoCollection("users") {

  // a JSONCollection is available here. 
  // It's more simple, easy to read and productive. 

  // No more '...map(_[...]("...")])' to read.
  // 99% of time, all I want is an access to my collection. Nothing else.
}

Providing a trait is easy to read and more productive.

What would be provided by such trait ?

just a collection: Future[JSONCollection]. This one is free. This makes the code more easy to read, nothing else: readability experience. 

A collection DI doesn't look really useful for me. Maybe a mixin trait ...
 

 

 

- readability (query): 

same reason above. map and flatMap...

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection.flatMap { _
  .find(selector)
  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))

  .cursor[User](ReadPreference.Primary)
  .collect[List](Int.MaxValue, Cursor.FailOnError[List[User]]())
}

//
// WHY NOT (ALSO) ?
// - no flatMap

That's async monadic reference. It could be replaced by a builder/factory, but I doubt it would bring improvement (and won't follow monadic rules).

 

// - no .options

?

no need to bring all options inside a method .options:

  .sort(sort)
  .options(QueryOpts(skipN = ((page - 1) * pageSize), batchSizeN = pageSize))
  .cursor[User](ReadPreference.Primary)

 

That's already the case with QueryOpts, even callable as QueryOpts().

 

 

// - no QueryOpts (no import needed...)

?

Readability experience. QueryOpts makes reading the code boring. Why not?

  .sort(sort)
  .options(
    skip = 100,
    limit = 100))
  .cursor[User](ReadPreference.Primary)

 

Can have a look.

 

 

// - a ctrl-spc inside an IDE list all options available to the developer (no need to search in the RM docs)

? 

// 

def find(selector: JsObject, sort: JsObject, page: Int, pageSize: Int) = collection
  .find(selector)
  .sort(sort)
  .skip((page - 1) * pageSize)

  .limit(pageSize)

  .cursor[User]()
  .collect[List]()

chaining .find .sort .skip .limit is more easy and confortable to read.

QueryOpts.{skip,batchSize,flags,tailable,...} are provided. Gathering everything (mixing option, cursor resolution, sort clause) at the same QueryBuilder level doesn't sound great for me.

For the developer, it's easier.

Opinionated

 

Inside my IDE I just have to use the shortcut "ctrl-space" to list all available methods.
No need to switch to my brower and open the RM docs.

?

 

- readability (json / bson DSL): 

a DSL could make the query more natural and easy to read (ReactiveMongo-Extensions provided an awesome json/bson DSL).

def findInText(text: String) = find(Json.obj(
    f"$$text" -> Json.obj(
      f"$$search" -> text)))

//
// WHY NOT (ALSO)?
//

def findInText(terms: String) = collection.find($text(terms))

As already discussed in this MailingList, ReactiveMongo-extensions is no longer maintained.

Tier project project are welcome/free to provide such DSL, having the core dev maintaining one is not a priority.

a project "reactivemongo-dsl" could be hosted inside the RM organization? 

By tier project I mean project outside the core organization.

 

 

 

- readability (criteria DSL): 

No need to talk more about that, there is an awesome project that provides a criteria DSL.

Conclusion

RM is awesome but it's a driver which provides core features at low level. With helpers. I get it. 

But when I create a new Scala+Mongo project I ask myself: is there a new library that provides the 3 golden features (top performance, high productivity and awesome readability)? RM is on top of the list but has a big lack on productivity and readability.

So, does the core RM developers want to make only a driver OR the most awesome Scala Mongo library for developers?

There is already a lot to do for the core dev, so yes the driver and the core lib (BSON/JSON support) are the priorities.

Developing the ecosystem with tier projects is welcome.

Nice to hear that :) Last question. Why ReactiveMongo-Extensions which provide helpers to be productive (DAO, DSL, Criteria) is no longer maintained?

ReactiveMongo is a volunteer project. If having time, the core dev would be happy to work on a lot more libraries (e.g. JSON4S, Kamon monitoring, Spark integration), there are a lot a interesting development that could be done. But priorities must be considered, to properly develop the core components rather than trying to do "everything" with a bad quality.

ReactiveMongo-Extensions was providing non-core features, partially now provided by either other tier/community libraries or by the new versions of the drivers.

Best regards.

[Chrissom]

As I understand, you are focusing on the core features in order to provide quality. Tiers projects are outside of the organization. Is it possible to have a list of them on the readme?

What's your current RM need? 

[Cédric Chantepie]

https://github.com/ReactiveMongo/reactivemongo-site/blob/gh-pages/ecosystem.md#third-party

[Chrissom]

I guess you don't have need.

Thanks for your time.

Le jeudi 11 mai 2017 22:28:02 UTC+2, Cédric Chantepie a écrit :

https://github.com/ReactiveMongo/reactivemongo-site/blob/gh-pages/ecosystem.md#third-party

[Cédric Chantepie]

.find(..).skip(..) and same for other QueryOpts.

[Cédric Chantepie]

On Thursday, 11 May 2017 22:43:34 UTC+2, Chrissom wrote:

I guess you don't have need.

Core dev cannot be integrated so easily. Volunteers for already discussed fixes or features are welcome (with prio discussion on the MailingList).

[Cédric Chantepie]

just a collection: Future[JSONCollection]. This one is free. This makes the code more easy to read, nothing else: readability experience. 

A collection DI doesn't look really useful for me. Maybe a mixin trait ...

https://github.com/cchantep/Play-ReactiveMongo/blob/5445835e22e1180bbb57b35eae85206785d87176/src/main/scala/play/modules/reactivemongo/WithCollection.scala#L34

[Cédric Chantepie]

Same for PR to improve the doc (as for the ecosystem listing).