Clojure concurrency with agents – no more explicit Threads

Let me state it now before it gets too late and I get blamed for spreading FUD or alike – I’m no expert at concurrency and have had very little to no exposure to the area. It’s been on my todo list for way too long and when I’ve heard about Clojure and its attempt to ease concurrency, my endeavor to learn both begun. It’s by no means a complete treatment of Clojure concurrency or a part thereof. Use it at your risk.

While looking for an example of how to spin off a thread in Clojure I came across the ant colony simulation of Rich Hickey – the creator of Clojure. It’s been when I made it to How does one start a thread in Clojure? at StackOverflow which turned out to be exactly the question I’d been asking myself for so long.

I’ve been reading about Clojure concurrency and its novel approach to the topic, and decided to give it a try. Read Concurrent Programming if you want a very quick overview of possible solutions in Clojure regarding concurrency. I worked up a real appetite for Clojure concurrency when I stumbled upon the question at StackOverflow – What is your opinion on Clojure?.

You : Can I use this small Java library called Clojure?
Boss: Why do you need it?
You : For some concurrency improvements.
Boss: Ok.

If, for some reason, you don’t like Clojure for it being a Lisp clone (which to many means nothing more than lots of parens), you should not cross it out and forgo the pleasure to run it as “a Java library for some concurrency improvements”. It will relieve the pain of doing it yourself, not necessarily right unless you’re really good at concurrency.

First things first – Clojure functions “implement the Java Callable, Runnable and Comparator interfaces.” It’s the very early exposure of yours to concurrency in Clojure regardless of your needs – whenever you define a function in Clojure, you actually instantiate a java.lang.Runnable and whenever you run a function, you in turn spin off a thread. Easy, isn’t it?

With that said, running a thread is as easy as defining a function with the defn macro and passing it on to the constructor of java.lang.Thread. It couldn’t be easier, could it?

user=> (defn f [] (println "I've been called"))
user=> (.start (Thread. f))
I've been called

You may also want to run it with the special form #() that creates an anonymous, inline function.

user=>  (.start (Thread. #(println "Another function, another message")))
Another function, another message

If that’s be all about concurrency in Clojure, I’d bet you’d definitely not call it a real contender for your time. Clojure gets further – it assumes that java.lang.Thread is a low-level artifact and helps you with its higher-level concurrency abstractions.

Clojure concurrency is based upon three higher-level abstractions – refs, agents and atoms. As I said earlier, I’m no expert in concurrency in general as well as Clojure’s approach so I leave it to you to find out more about them in their documentation at Clojure’s website.

My exposure to Clojure concurrency begun with agents. Dunno why I picked it up first, but it could be they’re the simplest abstractions of the trio. Let’s take a look at a possible use case – a way to record a message in a log file. It’s not using a real log file, but a structure to mimic one – an array. Without much success of a better example the one seemed fine to introduce agents with. The code should be self-explanatory. Ping me if it’s not (hint: the comments section).

user=> (def logs (agent []))
user=> @logs
user=> (import java.util.Date)
user=> (.. (Date.) (toString))
"Mon Jan 24 16:38:13 CET 2011"
user=> (defn current-time [] (.. (Date.) (toString)))
user=> (format "Executed %s at %s" (Thread/currentThread) (current-time))
"Executed Thread[main,5,main] at Mon Jan 24 16:38:24 CET 2011"
user=> (defn executed-at-msg [] (format "Executed %s at %s" (Thread/currentThread) (current-time)))
user=> (executed-at-msg)
"Executed Thread[main,5,main] at Mon Jan 24 16:38:36 CET 2011"
user=> (defn add-msg-to-agent [a] (conj a (executed-at-msg)))

The a – the input parameter of add-msg-to-agent function is for agent or array whatever makes more sense to you – in this particular case there’s no difference.

As far as I could understand agents in Clojure – they serve as a way to guard a structure, no matter what type, and a return value of a function given in a send or send-off becomes a new value of the agent. Clojure will take care of the function call at a suitable time – it’s not us who should synchronize access to the shared structure but Clojure itself and that’s how it eases our concurrent programming – we don’t have to deal with low-level details of handling concurrency ourselves. Just send or send-off a function and have its return value be a new value for the shared structure. Easy again, isn’t it?

user=> (send logs add-msg-to-agent)
#<Agent@ad72200: ["Executed Thread[clojure-agent-send-pool-0,5,main] at Mon Jan 24 16:39:14 CET 2011"]>
user=> @logs
["Executed Thread[clojure-agent-send-pool-0,5,main] at Mon Jan 24 16:39:14 CET 2011"]

It doesn’t however answer the question how to run a thread to send or send-off a value to a agent. That’s what I was looking for analyzing the ant colony simulation. It’s interesting to note that the entire application deals with concurrency with no Thread explicitly spun off (!) I first found it out in the aforementioned question at StackOverflow – How does one start a thread in Clojure?Jörg W Mittag wrote:

If you look at the canonical concurrency example in Clojure, Rich Hickey’s ant colony simulation, you will see that is uses exactly 0 threads. The only reference to java.lang.Thread in the entire source is three calls to Thread.sleep, whose sole purpose is to slow the simulation down so that you can actually see what is going on in the UI.

All the logic is done in Agents: one agent for every ant, one agent for the animation and one agent for the pheromone evaporation. The playing field is a transactional ref. Not a thread nor lock in sight.

For some time, I couldn’t understand how it’s done until I spot the solution – agents are sent a new value by means of a function that repeatedly calls itself (until it’s said to finish with a boolean var, albeit I couldn’t find a reference it’s done at all in the simulation). Utterly insane in its beauty. The trick is to use the do* macros like dotimes to fire up functions that will do the rest.

user=> (dotimes [_ 10] (send logs add-msg-to-agent))
user=> @logs
["Executed Thread[clojure-agent-send-pool-0,5,main] at Mon Jan 24 16:39:14 CET 2011" "Executed Thread[clojure-agent-send-pool-1,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-2,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-3,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-0,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-0,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-0,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-0,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-1,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-1,5,main] at Mon Jan 24 16:39:37 CET 2011" "Executed Thread[clojure-agent-send-pool-1,5,main] at Mon Jan 24 16:39:37 CET 2011"]

If the add-msg-to-agent function were to call itself it would create a thread that would run until it’s said to stop, e.g. with a boolean var. A very naive solution might be as follows – note the use of *agent* which is a reference to “the agent currently running an action on this thread, else nil”.

user=> ;; define two agents for a boolean var and a counter of how many times the agent is called
user=> (def running (agent true))
user=> (def counter (agent 0))
user=> ;; define a function to increment the counter agent's value
(defn increment-until-running-becomes-false [a]
  (when @running
    (send-off *agent* #'increment-until-running-becomes-false))
  (. Thread (sleep (* 1000 2)))
  (inc a))
user=> ;; fire it up
user=> (send counter increment-until-running-becomes-false)
#<Agent@4235e6e3: 0>
user=> ;; check the counter agent's value - it should be more than 0 after a few seconds
user=> @counter
user=> ;; shut the counter agent down - no more counter changes
user=> (send running (fn [a] false))
#<Agent@46dd75a4: true>
user=> ;; no more increments to the counter agent
user=> @counter
user=> @counter

I’m very pleased with the simplicity offered by Clojure. It takes time to fully comprehend its beauty, but I believe it ought to be very useful once you’re past the initial troubles. I do hope I’ll get better understanding of Clojure itself once I’ve done with the ant colony simulation. As with something new in our IT industry, the main trouble is where and how to use it effectively. Can you think of a better example to get the gist of Clojure concurrency abstraction and possible real-life use cases? I’m working on a GUI application in Clojure and would love using the machinery for such utterly insane tricks.

Be Sociable, Share!
This entry was posted in Languages.

2 Responses to Clojure concurrency with agents – no more explicit Threads

  1. Konrad Garus says:

    Looks like you could use Mark Volkmann’s introduction to Clojure. Take a look at agents here: Your intuitive and detailed description is a great complement to it, keep up the good work.

    Ideas: Anything dealing with simulations or numerical methods (from gazelle fleeing from leopard through physics). Web scraper or feed consumer. Games…

    • Jacek Laskowski says:

      I read it a couple of times, but it was early in my Clojure development days so time to read it again. Thanks for the hint re the Mark’s article and possible uses for agents.

Leave a Reply

%d bloggers like this: