Repository pattern implementation, thoughts, help

[Larry Weya]

I'm currently re-implementing part of a Laravel(PHP) codebase in Elixir where I had used the repository pattern in only some of the data models. In the Elixir implementation, I'd like to use it across all the data models mainly because

1. I can swap out the implementation within tests

2. I can swap out the implementation to e.g. go through a cache layer for reads

3. I want to able to use different ecto repos for e.g. read and write and having abstract repositories would allow this to be done from a central place

I worked on an initial implementation this weekend that works like this.

For each data entity, implement a "behaviour" with callbacks for that particular entity e.g. a UserRepository where you need to query for a list of active users would look like this

defmodule UserRepository do
  use Repository.Base, # parses opts and config and sets @repository to configured repository module
    otp_app: :my_app
    repository: MyApp.Repositories.EctoUserRepository # this points to the concrete implementation and should be set within the config

 
  @callback save(Ecto.Changeset.t) :: {:ok, User.t} | {:error, Ecto.Changeset.t}
  
  @callback get_active(non_neg_integer, non_neg_integer) :: [User.t]

end

The Ecto backed repository implementation would look like this

defmodule EctoUserRepository do
  use Repository.Ecto.Base, # parses the opts and config, sets @repo and @read_repo and implements a save function thats common to all Ecto repos
    repo: MyApp.Repo,
    read_repo: MyApp.Repo # could set a different repo for reads
 
  @behaviour UserRepository

  import Ecto.Query, only: [from: 2]

  def get_active(offset, limit) do
    (from u in User,
      where: u.active == true,
      offset: ^offset,
      limit: ^limit)
    |> @read_repo.all()
  end
end

Please share any thoughts on this.

Now this is where I need some help, I want to able to use the repository as

UserRepository.get_active(0, 20)

what I've done at the moment is redefine each function within the UserRepository and delegate to whatever repository is configured

defmodule UserRepository do
  ...

  
  @callback save(Ecto.Changeset.t) :: {:ok, User.t} | {:error, Ecto.Changeset.t}
  
  @callback get_active(non_neg_integer, non_neg_integer) :: [User.t]

    def save(changeset) do

    @repository.save(changeset)
  end

  def get_active(offset, limit) do
    @repository.get_active(offset, limit)
  end
end

I find that this is quite repetitive and forces the same for all repository implementations and functions.

[José Valim]

I believe the question you need to ask yourself is of you really need this abstraction in the first place. You are now using a new platform but you are worried about concerns that are more relevant in the platform you originally came from.

First of all, I don't believe in swapping the implementation in tests as a general pattern/feature. Every time you swap, you still need to write an integration test to guarantee the two layers work together, so what are you gaining? If you are going to swap only in some cases, I would rather define proper contracts for those particular cases instead of swapping an implementation detail (I wrote about this in Plataformatec blog, I am on my phone right now but the digest is that, for an Twitter http client, you mock the "get_tweets" functionality and not the Http client itself).

Similarly concerns like caching and tests performance are going to hit you much later in Elixir and you can still provide that at the Ecto repository level instead of having to write your own user repositories.

It is really common for OO developers to think of Elixir modules as the minimal code structure (and therefore breaking everything into modules) and forget about functions. My advice would be to think of those as simply regular functions that are grouped based on functionality rather than on structure, for example MyApp.Accounts.get_active_users. 

--
You received this message because you are subscribed to the Google Groups "elixir-lang-talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-ta...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/47df7f07-0e79-40d7-b551-86839dba22b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

José Valim

www.plataformatec.com.br

Skype: jv.ptec

Founder and Director of R&D

[Larry Weya]

As always, thanks for taking the time to reply.

On Sunday, 12 June 2016 13:16:17 UTC+3, José Valim wrote:

I believe the question you need to ask yourself is of you really need this abstraction in the first place. You are now using a new platform but you are worried about concerns that are more relevant in the platform you originally came from.

 

Its not really a concern that was only relevant for the other platform (its in fact not built in and had to build it in), its just a general design pattern thats meant as a guide so you don't have e.g. queries that do the same things duplicated across the codebase. So instead of multiple controller functions composing different variations of the "get active" query, you have in a central place. 

First of all, I don't believe in swapping the implementation in tests as a general pattern/feature. Every time you swap, you still need to write an integration test to guarantee the two layers work together, so what are you gaining? If you are going to swap only in some cases, I would rather define proper contracts for those particular cases instead of swapping an implementation detail (I wrote about this in Plataformatec blog, I am on my phone right now but the digest is that, for an Twitter http client, you mock the "get_tweets" functionality and not the Http client itself).

 

I had read the article and coincidentally I was mocking http requests and having a difficult time. The article guided my design into implementing a test-only interface that exposed the required functionality. The article actually also also guided part of my repository implementation and I believe its in the same spirit where when testing a controller function, I can use an in-memory repository that satisfies the interface and use the Ecto based repository in production. The whole point of the abstraction for me is to keep the implementation details of how data is stored/retrieved outside of my controller functions. 

Similarly concerns like caching and tests performance are going to hit you much later in Elixir and you can still provide that at the Ecto repository level instead of having to write your own user repositories.

Performance, especially within tests is not as much of a concern as simplicity and I would argue that for the same reason you would have an in-memory Twitter client, you could have an in-memory repository within your tests.

It is really common for OO developers to think of Elixir modules as the minimal code structure (and therefore breaking everything into modules) and forget about functions. My advice would be to think of those as simply regular functions that are grouped based on functionality rather than on structure, for example MyApp.Accounts.get_active_users. 

I think this is an approach that could have the same result, keeping the underlying queries and Ecto interactions outside of the controllers (and other layers where we want to control how the data is accessed). I do however feel that the repository pattern provides a convention on how the code should be structured but perhaps my implementation is a bit over-engineered. I will go back and look at how it can be simplified.

To unsubscribe from this group and stop receiving emails from it, send an email to elixir-lang-talk+unsubscribe@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/elixir-lang-talk/47df7f07-0e79-40d7-b551-86839dba22b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Larry Weya]

To be clear, this is not about swapping out Ecto.Repo for another Repo implementation, so perhaps repository is not the best name. Its about having a behavior for your data entity that is satisfied by a concrete implementation that uses Ecto and another that could use a List.

[José Valim]

To be clear, this is not about swapping out Ecto.Repo for another Repo implementation, so perhaps repository is not the best name. Its about having a behavior for your data entity that is satisfied by a concrete implementation that uses Ecto and another that could use a List.

I understood that and I wrote my original e-mail with exactly that in mind. :) When you say "having a behavior for your data entity that is satisfied by a concrete implementation", it still feels like you are applying patterns from other paradigms and platforms without taking into account the options provided by Elixir.

Again, if you want to replace an implementation that uses Ecto by another that uses a list, I would specially handle those cases by defining a proper behaviour instead of simply having it as a default.

 

Performance, especially within tests is not as much of a concern as simplicity and I would argue that for the same reason you would have an in-memory Twitter client, you could have an in-memory repository within your tests.

The biggest point about replacing Twitter is to avoid an external dependency you can't control in your tests. Otherwise your tests will fail for reasons beyond your control. Unless the whole purpose of your application is to integrate with Twitter, I wouldn't compare it to my data layer.

 

I think this is an approach that could have the same result, keeping the underlying queries and Ecto interactions outside of the controllers (and other layers where we want to control how the data is accessed).

Right and that's what I am trying to say. You don't need the "repository pattern". You need modules and functions. Organizing everything into patterns that are tied more to the structure is going to constrain you in the long term. I realize I am repeating myself so I will refrain from commenting again. All of the warnings were in the original e-mail. :)

[Larry Weya]

I've gone back to the simpler approach you suggested where I have a module - UserStore that exposes the required functions and uses an Ecto.Repo internally.

Thanks again for the feedback.