first time without state - and I'm lost

[Scaramaccai]

Hi everyone,

I wanted to give a try to Clojure and functional programming in general but I can't really stop thinking "object oriented" or "with state everywhere". After 20+ years with objects + state I guess I'm lost without them :)

The first thing I want to try is to get some data from an API that needs OAuth authentication. I would like to hide the fact that there's an OAuth token to be sent. In pseudo-Java I would do something like:

class MyAuthHttpClient {
  private token;
  public MyAuthHttpClient(String usr, String psw) {...}

  public ... getResponse(Url) {
  // here check if token is available and if it is expiring;
  // if expiring -> fetch a new token before call the http service
  // caller doesn't even know there's a token involved in the process
  }
}

What's the proper way to do that in Clojure?

I guess one way would be to have a function that returns a new token given a (possibly old) token and user+psw

(defn gettkn [usr, psw, tkn] (return a new token if tkn is expiring or tkn if not expiring))

(def wrap-gettkn (partial gettkn "myuser" "mypass"))

(defn geturl [url, tkn] client/get url {:oauth-token (wrap-gettkn tkn)})


I can "save" usr and psw, but I always have to "keep" the tkn around at every level;
while I would like the token to be "hidden" to the "geturl" clients (just like I did in the "private token" in the pseudo-Java).

What's the proper way of doing something like this in Clojure?

[Orestis Markou]

I’m interested in this too — you can get inspiration from https://github.com/cognitect-labs/aws-api/blob/master/src/cognitect/aws/credentials.clj which does something similar for expired credentials. Seems like a mutable internal field is fine for this use case.

--
You received this message because you are subscribed to the Google
Groups "Clojure" group.
To post to this group, send email to clo...@googlegroups.com
Note that posts from new members are moderated - please be patient with your first post.
To unsubscribe from this group, send email to
clojure+u...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.
To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/6aa66613-24d9-4ebc-87d4-e9a6cca05165%40googlegroups.com.

[Jag Gunawardana]

I know that lost feeling. I came from a background of OO too. Most of the languages I'd used (C++, Java, Python, Go) had either total OO orientation, or at least pushed you into an OO way of thinking. I found that reading some Clojure books and other's code helped a lot (Joy of Clojure, Eric Normand's content etc). Once I got my head around some practical examples like the one you state, I found that my thinking changed. I also found the MIT SCIP fried the way I think and spat it out anew. It takes time and practice (well at least it took me time and practice).

I think that what you are doing is similar to something I did to get the Google Identity Keys. 

(defmethod ig/init-key ::get-google-keys [_ _]

  (let [gkeys (ref {:keys nil :expires 0})]

    (fn []

      (let [current @gkeys]

        (:keys (if (>= (c/to-epoch (t/now)) (:expires current))

                 (do

                   (log/info "Fetching Google Keys, expiry: " (:expires current))

                   (dosync

                    (ref-set gkeys (fetch-google-keys ::jwk)))) ;; only use JWK for now

                 current))))))

I was using the excellent Integrant library, but it would work the same without it. I guess that you could do something similar with your token, if it had expired, then it would fetch it?

The main thing that I changed in my mind is that it is ok to pass the state to a function and get back a response, hiding everything in private data doesn't give you as much as OO promises it will.

clo...@googlegroups.com

For more options, visit this group at
http://groups.google.com/group/clojure?hl=en
---
You received this message because you are subscribed to the Google Groups "Clojure" group.

To unsubscribe from this group and stop receiving emails from it, send an email to clo...@googlegroups.com.

[Gerard Klijs]

What helped for me was picking an existing Clojure project, and try to make changes to it. I also struggled with some things. And not declaring and mutating objects was confusing at times. But I think with this regard it helped me a lot. In my case it was an existing cljs snake game https://github.com/OMantere/cljs-snake and add multi player with a clj backend to it.

It's just an idea, but I think this might work better than trying to convert existing oop code.

[Matthew Downey]

I totally understand that, I felt the same way first coming to Clojure.

For this situation, there are a couple approaches that you could take

depending on how hardcore you want to be about keeping things functional

& immutable.

The most similar thing you could do to your Java code would be keeping

the token in an atom inside of a connection record.

  (defrecord MyAuthHttpClient [token user psw])


  (defn my-auth-http-client [usr psw]
    (let [token (atom nil)]
      (maybe-update-token token usr psw)
      (->MyAuthHttpClient token usr psw)))


  (defn get-response [client url]
    (maybe-update-token (:token client) (:user client) (:psw client))
    (let [token @(:token client)]
      (do-request token url)))

If you want to do the same thing, but are uncomfortable storing username

and password in the record, you can close them into a function:

  (defrecord MyAuthHttpClient [token refresh-token])


  (defn my-auth-http-client [usr psw]
    (let [refresh-token (fn [t] (maybe-update-token t usr psw))
          token (atom nil)]
      (refresh-token token)
      (->MyAuthHttpClient token refresh-token)))


  (defn get-response [client url]
    ((:refresh-token client) (:token client))
    (let [token @(:token client)]
      (do-request token url)))

If you want polymorphism, such that you can have many authentication schemes

for different HttpClients, and insulate your calling code from those details,

use a protocol:

  (defprotocol Client
    (get-response [this url] "Perform an HTTP GET against `url`."))


  (defrecord NoAuthClient []
    Client
    (get-response [this url] (slurp url)))


  ;; E.g.
  (get-response (->NoAuthClient) "https://blockchain.info/stats?format=json")
  ;=> "{\"timestamp\":1.589365823E12,\"market_price_usd\":8920.2,..."


  (defrecord MyAuthHttpClient [token refresh-token]
    (get-response [this url]
      (refresh-token token)
      (do-request token url)))

So far, this has all been pretty a pretty typical OO style of abstraction &

separation of concerns. For certain problems, this is really effective.

Now let's imagine that your application grows a lot, you have a dozen

different http clients, and they're all hitting endpoints with different rate

limits many times per second. You want to add a rate limiting layer, simplify

logging of all of the requests / responses, and make the interface

asynchronous for callers so that they're not blocking for every response.

In that case (in my option), the most effective thing is to make your client 

more functional — it shouldn't keep state, perform side effects, etc. All of 

that would happen at a higher level of your code, and give you the flexibility 

to combine different clients with the rate limit layer and the async request

code however you want. 

Instead, you client would return information about the requests that it needs 

made. This is a stronger abstraction with stronger separation of concerns —

the request generation & signing code don't care how or when the requests are 

executed, the requesting code doesn't care how requests are built or signed,

and the highest-level code only knows that it made a request — but it's 

significantly more work to define something that precisely.

  

E.g. you might have:

  (defprotocol Client
    (request [this request-data]
      "Build a request and return `this`.")


    (response [this response-data]
      "Return a possibly updated `this` for a response to one of the
      client's requests.")


    (consume-requests [this]
      "Consume all queued requests, return [this requests]."))


  ;; The NoAuthClient doesnt do much...
  (defrecord NoAuthClient [queued]
    Client
    (request [this request-data] (assoc this :queued (conj queued request-data)))
    (response [this _] this)
    (consume-requests [this] [(assoc this :queued []) queued]))


  ;; Your token auth client, on the other hand...
  (defrecord MyAuthHttpClient
    [token refresh-token requests queued-for-after-refresh]


    Client
    (request [this request-data]
      ;; It is not actually making any requests to refresh, just
      ;; figuring out _what_ needs to be done, and letting the calling
      ;; code take care of _how_ to do it (via clj-http?, aleph?, right
      ;; away?, later?, etc)
      (if (token-expired-given-current-time? token (:ts request-data))
        (-> this
            (update :queued-for-after-refresh conj request-data)
            (update :requests conj (refresh-token token)))
        (update this :requests conj request-data)))


    (response [this response-data]
      (if (is-token-refresh-response? response-data)
        (let [new-token (parse-new-token response-data)

              ;; Any requests that were awaiting a new token are
              ;; now ready, but probably need to have the token
              ;; included
              insert (map #(assoc % :token new-token))
              requests (into requests insert queued-for-after-refresh)]
          (assoc this :token new-token
                      :requests requests
                      :queued-for-after-refresh []))
        this))


    (consume-requests [this]
      [(assoc this :requests []) requests]))

[James Reeves]

On Wed, 13 May 2020 at 12:16, Matthew Downey <matthew...@gmail.com> wrote:

The most similar thing you could do to your Java code would be keeping

the token in an atom inside of a connection record.

As a quick note, if you don't need polymorphism, maps should be favoured over records.

--

James Reeves

booleanknot.com

[J.-F. Rompre]

Hi @Scaramaccai,

If you are starting out, it's always best to keep things as bare as possible until you discover a better way.

If as you say you can save the usr/pwd, and the decision to keep or replace the token is entirely left to the token provider, then I think something like the following may be suitable.

For anyone interested in learning the ins and outs of OO vs Functional, static typing vs. dynamic, I highly recommend the Programming Languages Specialization on Coursera.

(defonce user-tokens (atom {:users {}))

(defn get-updated-token [usr pwd tok]
    ;; Returns tok if still valid or a new token
)

(defn get-token [usr, psw]
  (let [f (fn f [tok]
              [tok, (fn [] (f (get-updated-token usr psw tok)))])
        path [:users usr]
        thunk (or (get-in @user-tokens path) (f nil))
        [token thunk'] (thunk)]
    (do (swap! user-tokens assoc-in path thunk')
        token)))

(defn geturl [url]
  (client/get url {:oauth-token (get-token "user" "pwd")}))

[Gary Johnson]

Hi Scaramaccai,

Several posters in this thread have given you example code for how

to solve your OAuth token problem in Clojure. Perhaps I can add to

this discussion by pointing out the fundamentals of functional

programming that can help you decide how to solve problems like

this.

Loops -> Recursions and State -> Bindings

You asked how to handle state in a functional programming language.

In some situations, it may be easiest to just store stateful values

in a mutable container type like a ref, agent, or atom. This is not

a strictly functional approach, but these are tools that Clojure

provides to make a variety of programming tasks somewhat easier (in

the Rich Hickey sense of the word).

In many cases, this is neither necessary nor preferable. Instead,

you can use the functional programming approach of emulating state

change over time by passing your new derived values as arguments to

the next step in a recursive function.

That is, in FP "state change" happens on the stack (through

function bindings and let bindings) rather than on the heap

(through direct assignment to mutable containers).

Consider these two approaches to computing the factorial function.

1. Imperative (loops + mutation)

(defn fact-imp [n]
  (let [result (atom 1)]
    (dotimes [i n]
      (swap! result * (inc i)))
    @result))

2. Functional (recursion + fn bindings)

(defn fact-rec [n]
  (if (<= n 0)
    1
    (* n (fact-rec (dec n)))))

These two implementations will return the same outputs for the same

inputs. Note that in the functional approach, fact-rec computes the

next value of the result and passes it as the input to itself

rather than mutating a local variable as in the imperative case

with fact-imp.

Savvy readers will notice that fact-rec is not tail recursive and is

therefore prone to stack overflow for large values of n. Rewriting

it to work with loop+recur is left as an exercise for the reader. ;)

The approach shown above is a typical solution for state that goes

through a series of intermediate changes to produce a final result.

A similar approach for the same issue is to model all the values

that you would have stored one at a time in a stateful variable as

an immutable sequence of values. This approach relies on lazy

evaluation.

To illustrate, I will once again implement factorial using this

technique.

3. Functional (sequences + lazy evaluation)

(defn fact-lazy [n]
  (letfn [(next-step [[i x]] [(inc i) (* x (inc i))])
          (fact-seq [pair] (lazy-seq (cons pair (fact-seq (next-step pair)))))]
    (second (nth (fact-seq [0 1]) n))))

In this example, next-step derives the next state value from the

current one. The fact-seq function returns a lazy sequence of all

the [i factorial(i)] pairs from [0 1] to [infinity

factorial(infinity)]. This sequence is obviously never fully

realized since it would throw your application into an infinite

recursion. We then use nth to grab the [n factorial(n)] pair off of

the lazy sequence and second plucks out just factorial(n) to return

as the result of our fact-lazy function.

Once again, I never mutated any variables in place. I simply

created a recursive algorithm that could derive the next value from

the current value. Unlike the non-tail-recursive fact-rec above,

this lazy sequence implementation is immune to stack overflow

errors. However, fact-lazy will use more memory and more CPU cycles

than fact-rec's eager implementation because it has to create and

release the lazy sequence and intermediate vector pairs. These are

all tradeoffs, which you would need to consider in determining the

approach that might work best for your problem.

Hiding State

In the three approaches I showed to represent state change in your

Clojure program, none of these went out of their way to either hide

or share the state values over time. Here, you again have broadly two choices:

1. Global mutable variables

If you don't need to hide your application state, the best approach

is just to store it in a global mutable container like a ref, agent,

or atom. Then you can introspect or manipulate it from your REPL,

and multiple functions in your program can all access it as needed.

(def state (atom 0))

2. Closures

If you need to hide your application state for some reason (and

there are often less reasons to do this than you might imagine),

then your best friend is a closure function. A closure is a

function which "closes over" the bindings which exist when it is

defined by capturing references to them in its free variables

(i.e., variables which are not bound as function args or let args

in the function body).

Functions that return closures around mutable state can work

a bit like OOP constructor functions, creating a function that

behaves a bit like an object with internal stateful attributes.

Welcome to OOP inverted!

(defn get-token
  "Somehow get an OAuth token for this user+pass combination."
  [user pass]
  {:token   "some-token" ; replace this with something real
   :expires (+ (System/currentTimeMillis) 1000000)})

(defn expired?
  "Returns true if the token has expired."
  [token]
  (> (System/currentTimeMillis) (:expires token)))

(defn make-http-client
  "Return a closure that retains the token associated with this
  user+pass combination and can make HTTP requests using it."
  [user pass]
  (let [token (atom (get-token user pass))]
    (fn [url]
      (when (expired? @token)
        (reset! token (get-token user pass)))
      (client/get url {:oauth-token (:token @token)}))))

(def get-response (make-http-client "my-user" "my-pass"))

(get-response "some-url")
(get-response "another-url")
(get-response "yet-another-url")
(get-response "even-another-url")

The code above creates a closure function called get-response that contains an internal reference to a token atom. This atom is automatically updated using the stored user and pass whenever the token expires. All HTTP requests generated by get-response include the up-to-date OAuth token as expected.

I hope this exposition on some of the approaches used to solve problems in a functional programming style will help you both in your current project and many more going forward. Just remember to visualize state change over time as a sequence of values which can be recursively derived by their previous values. Whether the transformation from current value to next value is best done in an eager fashion (like fact-rec) or in a lazy fashion (like fact-lazy) will be determined by your particular problem domain. In many cases, there is not much value to hiding state, so if you choose to use mutable container types like refs, atoms, or agents, you are likely better off just making them globally available. However, if you really do want to capture state lexically within a function (perhaps because you need to pass it to another namespace), then closures can give you that ability in a powerful yet succinct way.

And to echo James Reeves' comments, your code will be shorter, simpler, and easier to extend if you just stick with plain maps. Records and protocols are great if you need fast polymorphism in your algorithm or if you need to extend a class that you don't control to implement one of your functions. However, they add a good bit of extra overhead to your code base that can make it more difficult to read, debug, and extend later.

Alright, that's enough from me on this topic. Go forth and write beautiful, elegant, functional code.

(hack :the-planet)

  ~Gary

[Ernesto Garcia]

Hi, it's a long time that this question was posted, but I have found it interesting in the implementation of token refreshes.

First of all, for service invocation, given a `revise-oauth-token` method, I think this is good client code:

(http/request
  {:method :get
   :url "https://example.com/"
   :oauth-token (revise-oauth-token token-store)})

If you find it too repetitive or fragile in your client code, you can make a local function, but I wouldn't abstract the service invocation at a higher layer.

Regarding the implementation of the token store, we could initially think of a synchronized store, like an atom, and `revise-oauth-token` would swap its content when a refresh is required. This is inconvenient for multithreaded clients, because there could be several refresh invocations going on concurrently.

In order to avoid concurrent refreshes, I propose to implement the token store as an atom of promises. Implementation of `revise-oauth-token` would be:

(defn revise-oauth-token [token-store]
  (:access_token
    @(swap! token-store
       (fn [token-promise]
         (if (token-needs-refresh? @token-promise (Instant/now))
           (delay (refresh-oauth-token (:refresh_token @token-promise)))
           token-promise)))))

Note that using a delay avoids running `refresh-oauth-token` within the `swap!` operation, as this operation may be run multiple times.

Also note that `token-needs-refresh` takes an argument with the present time. This keeps the function pure, which could help for unit testing, for example.

There is an alternative implementation using `compare-and-set!` that avoids checking `token-needs-refresh?` several times, but it is more complicated. I have posted full sample code in a gist: https://gist.github.com/titogarcia/4f09bcc5fa38fbdc1076954b9a99a8fc

Remark: None of this refers to "functional programming" per se. Dealing with state in a purely functional way involves using different constructs (like possibly monads, for which you can find Clojure libraries if you are interested), and best practices are still a topic of research. Clojure has taken the pragmatic approach of making purely functional code easy to write, but it doesn't reject the use of state, rather it provides well-behaved primitives like vars, atoms, agents, etc.

Ernesto

[Justin Smith]

The usage of delay here is clever. I suggest as an addition, using
`force` instead of `deref` to disambiguate delay vs. atom (of course
if you take a few moments to think about it, swap! shouldn't return an
atom etc., but I think it becomes clearer with force).

> --
> You received this message because you are subscribed to the Google
> Groups "Clojure" group.
> To post to this group, send email to clo...@googlegroups.com
> Note that posts from new members are moderated - please be patient with your first post.
> To unsubscribe from this group, send email to

> clojure+u...@googlegroups.com

> For more options, visit this group at
> http://groups.google.com/group/clojure?hl=en
> ---
> You received this message because you are subscribed to the Google Groups "Clojure" group.

> To unsubscribe from this group and stop receiving emails from it, send an email to clojure+u...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/ac79058b-2c31-4b9c-9cf3-e2de998eb8deo%40googlegroups.com.

[Ernesto]

That's nice. We could do something like:

(-> (swap! ...)

  force

  :access_token)

You received this message because you are subscribed to a topic in the Google Groups "Clojure" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/clojure/Vur5Lol45EE/unsubscribe.
To unsubscribe from this group and all its topics, send an email to clojure+u...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/clojure/CAGokn9L_Od2ZN2LJAsYUfJ2G_hbLKkamkUxgFX2vTKySxpHQWg%40mail.gmail.com.

[Oleksandr Shulgin]

On Mon, Jun 15, 2020 at 7:34 PM Ernesto Garcia <titog...@gmail.com> wrote:

Hi, it's a long time that this question was posted, but I have found it interesting in the implementation of token refreshes.

First of all, for service invocation, given a `revise-oauth-token` method, I think this is good client code:

(http/request
  {:method :get
   :url "https://example.com/"
   :oauth-token (revise-oauth-token token-store)})

If you find it too repetitive or fragile in your client code, you can make a local function, but I wouldn't abstract the service invocation at a higher layer.

Regarding the implementation of the token store, we could initially think of a synchronized store, like an atom, and `revise-oauth-token` would swap its content when a refresh is required. This is inconvenient for multithreaded clients, because there could be several refresh invocations going on concurrently.

In order to avoid concurrent refreshes, I propose to implement the token store as an atom of promises. Implementation of `revise-oauth-token` would be:

(defn revise-oauth-token [token-store]
  (:access_token
    @(swap! token-store
       (fn [token-promise]
         (if (token-needs-refresh? @token-promise (Instant/now))
           (delay (refresh-oauth-token (:refresh_token @token-promise)))
           token-promise)))))

Note that using a delay avoids running `refresh-oauth-token` within the `swap!` operation, as this operation may be run multiple times.

Also note that `token-needs-refresh` takes an argument with the present time. This keeps the function pure, which could help for unit testing, for example.

There is an alternative implementation using `compare-and-set!` that avoids checking `token-needs-refresh?` several times, but it is more complicated. I have posted full sample code in a gist: https://gist.github.com/titogarcia/4f09bcc5fa38fbdc1076954b9a99a8fc

I think it is worth mentioning an alternative approach to avoid concurrent token refresh may be to use a scheduled task to run shortly before the currently valid token is going to expire.

The http call then only needs deref and the background task can even use reset! on the atom, as there are no concurrent update operations to race against.

--

Alex