Scala Neophyte

Scala Neophyte http://danielwestheide.com Scala Neophyte The Neophyte's Guide to Scala Part 12: Type classes - Daniel Westheide http://danielwestheide.com/blog/2013/02/06/the-neophytes-guide-to-scala-part-12-type-classes.html <div class="entry-content">After having discussed several functional programming techniques for keeping things DRY and flexible in the last two weeks, in particular function composition, partial function application, and currying, we are going to stick with the general notion of making your code as flexible as possible.However, this time, we are not looking so much at how you can leverage functions as first-class objects to achieve this goal. Instead, this article is all about using the type system in such a manner that it’s not in the way, but rather supports you in keeping your code extensible: You’re going to learn about type classes.You might think that this is some exotic idea without practical relevance, brought into the Scala community by some vocal Haskell fanatics. This is clearly not the case. Type classes have become an important part of the Scala standard library and even more so of many popular and commonly used third-party open-source libraries, so it’s generally a good idea to make yourself familiar with them.I will discuss the idea of type classes, why they are useful, how to benefit from them as a client, and how to implement your own type classes and put them to use for great good.<h2>The problem</h2>Instead of starting off by giving an abstract explanation of what type classes are, let’s tackle this subject by means of an – admittedly simplified, but nevertheless resonably practical – example.Imagine that we want to write a fancy statistics library. This means we want to provide a bunch of functions that operate on collections of numbers, mostly to compute some aggregate values for them. Imagine further that we are restricted to accessing an element from such a collection by index and to using the <code>reduce</code> method defined on Scala collections. We impose this restriction on ourselves because we are going to re-implement a little bit of what the Scala standard library already provides – simply because it’s a nice example without many distractions, and it’s small enough for a blog post. Finally, our implementation assumes that the values we get are already sorted.We will start with a very crude implementation of <code>median</code>, <code>quartiles</code>, and <code>iqr</code> numbers of type <code>Double</code>:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 </pre></td><td class="code"><pre><code class="scala">object Statistics { def median(xs: Vector[Double]): Double = xs(xs.size / 2) def quartiles(xs: Vector[Double]): (Double, Double, Double) = (xs(xs.size / 4), median(xs), xs(xs.size / 4 * 3)) def iqr(xs: Vector[Double]): Double = quartiles(xs) match { case (lowerQuartile, _, upperQuartile) => upperQuartile - lowerQuartile } def mean(xs: Vector[Double]): Double = { xs.reduce(_ + _) / xs.size } } </code></pre></td></tr></table></div></figure>The median cuts a data set in half, whereas the lower and upper quartile (first and third element of the tuple returned by our <code>quartile</code> method), split the lowest and highest 25 percent of the data, respectively. Our <code>iqr</code> method returns the interquartile range, which is the difference between the upper and lower quartile.Now, of course, we want to support more than just double numbers. So let’s implement all these methods again for <code>Int</code> numbers, right?Well, no! First of all, we can’t simply overload the methods defined above for <code>Vector[Int]</code> without some dirty tricks, because the type parameter suffers from type erasure. Also, that would be a tiny little bit repetitious, wouldn’t it?If only <code>Int</code> and <code>Double</code> would extend from a common base class or implement a common trait like <code>Number</code>! We could be tempted to change the type required and returned by our methods to that more general type. Our method signatures would look like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 </pre></td><td class="code"><pre><code class="scala">object Statistics { def median(xs: Vector[Number]): Number = ??? def quartiles(xs: Vector[Number]): (Number, Number, Number) = ??? def iqr(xs: Vector[Number]): Number = ??? def mean(xs: Vector[Number]): Number = ??? } </code></pre></td></tr></table></div></figure>Thankfully, in this case there is no such common trait, so we aren’t tempted to walk this road at all. However, in other cases, that might very well be the case – and still be a bad idea. Not only do we drop previously available type information, we also close our API against future extensions to types whose sources we don’t control: We cannot make some new number type coming from a third party extend the <code>Number</code> trait.Ruby’s answer to that problem is monkey patching, polluting the global namespace with an extension to that new type, making it act like a <code>Number</code> after all. Java developers who have got beaten up by the Gang of Four in their youth, on the other hand, will think that an Adapter may solve all of their problems:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 </pre></td><td class="code"><pre><code class="scala">object Statistics { trait NumberLike[A] { def get: A def plus(y: NumberLike[A]): NumberLike[A] def minus(y: NumberLike[A]): NumberLike[A] def divide(y: Int): NumberLike[A] } case class NumberLikeDouble(x: Double) extends NumberLike[Double] { def get: Double = x def minus(y: NumberLike[Double]) = NumberLikeDouble(x - y.get) def plus(y: NumberLike[Double]) = NumberLikeDouble(x + y.get) def divide(y: Int) = NumberLikeDouble(x / y) } type Quartile[A] = (NumberLike[A], NumberLike[A], NumberLike[A]) def median[A](xs: Vector[NumberLike[A]]): NumberLike[A] = xs(xs.size / 2) def quartiles[A](xs: Vector[NumberLike[A]]): Quartile[A] = (xs(xs.size / 4), median(xs), xs(xs.size / 4 * 3)) def iqr[A](xs: Vector[NumberLike[A]]): NumberLike[A] = quartiles(xs) match { case (lowerQuartile, _, upperQuartile) => upperQuartile.minus(lowerQuartile) } def mean[A](xs: Vector[NumberLike[A]]): NumberLike[A] = xs.reduce(_.plus(_)).divide(xs.size) } </code></pre></td></tr></table></div></figure>Now we have solved the problem of extensibility: Users of our library can pass in a <code>NumberLike</code> adapter for <code>Int</code> (which we would likely provide ourselves) or for any possible type that might behave like a number, without having to recompile the module in which our statistics methods are implemented.However, always wrapping your numbers in an adapter is not only tiresome to write and read, it also means that you have to create a lot of instances of your adapter classes when interacting with our library.<h2>Type classes to the rescue!</h2>A powerful alternative to the approaches outlined so far is, of course, to define and use a type class. Type classes, one of the prominent features of the Haskell language, despite their name, haven’t got anything to do with classes in object-oriented programming.A type class <code>C</code> defines some behaviour in the form of operations that must be supported by a type <code>T</code> for it to be a member of type class <code>C</code>. Whether the type <code>T</code> is a member of the type class <code>C</code> is not inherent in the type. Rather, any developer can declare that a type is a member of a type class simply by providing implementations of the operations the type must support. Now, once <code>T</code> is made a member of the type class <code>C</code>, functions that have constrained one or more of their parameters to be members of <code>C</code> can be called with arguments of type <code>T</code>.As such, type classes allow ad-hoc and retroactive polymorphism. Code that relies on type classes is open to extension without the need to create adapter objects.<h3>Creating a type class</h3>In Scala, type classes can be implemented and used by a combination of techniques. It’s a little more involved than in Haskell, but also gives developers more control.Creating a type class in Scala involves several steps. First, let’s define a trait. This is the actual type class:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 </pre></td><td class="code"><pre><code class="scala">object Math { trait NumberLike[T] { def plus(x: T, y: T): T def divide(x: T, y: Int): T def minus(x: T, y: T): T } } </code></pre></td></tr></table></div></figure>We have created a type class called <code>NumberLike</code>. Type classes always take one or more type parameters, and they are usually designed to be stateless, i.e. the methods defined on our <code>NumberLike</code> trait operate only on the passed in arguments. In particular, where our adapter above operated on its member of type <code>T</code> and one argument, the methods defined for our <code>NumberLike</code> type class take two parameters of type <code>T</code> each – the member has become the first parameter of the operations supported by <code>NumberLike</code>.<h3>Providing default members</h3>The second step in implementing a type class is usually to provide some default implementations of your type class trait in its companion object. We will see in a moment why this is generally a good strategy. First, however, let’s do this, too, by making <code>Double</code> and <code>Int</code> members of our <code>NumberLike</code> type class:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 </pre></td><td class="code"><pre><code class="scala">object Math { trait NumberLike[T] { def plus(x: T, y: T): T def divide(x: T, y: Int): T def minus(x: T, y: T): T } object NumberLike { implicit object NumberLikeDouble extends NumberLike[Double] { def plus(x: Double, y: Double): Double = x + y def divide(x: Double, y: Int): Double = x / y def minus(x: Double, y: Double): Double = x - y } implicit object NumberLikeInt extends NumberLike[Int] { def plus(x: Int, y: Int): Int = x + y def divide(x: Int, y: Int): Int = x / y def minus(x: Int, y: Int): Int = x - y } } } </code></pre></td></tr></table></div></figure>Two things: First, you see that the two implementations are basically identical. That is not always the case when creating members of a type classes. Our <code>NumberLike</code> type class is just a rather narrow domain. Later in the article, I will give examples of type classes where there is a lot less room for duplication when implementing them for multiple types. Second, please ignore the fact that we are losing precision in <code>NumberLikeInt</code> by doing integer division. It’s all to keep things simple for this example.As you can see, members of type classes are usually singleton objects. Also, please note the <code>implicit</code> keyword before each of the type class implementations. This is one of the crucial elements for making type classes possible in Scala, making type class members implicitly available under certain conditions. More about that in the next section.<h3>Coding against type classes</h3>Now that we have our type class and two default implementations for common types, we want to code against this type class in our statistics module. Let’s focus on the <code>mean</code> method for now:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 </pre></td><td class="code"><pre><code class="scala">object Statistics { import Math.NumberLike def mean[T](xs: Vector[T])(implicit ev: NumberLike[T]): T = ev.divide(xs.reduce(ev.plus(_, _)), xs.size) } </code></pre></td></tr></table></div></figure>This may look a little intimidating at first, but it’s actually quite simple. Our method takes a type parameter <code>T</code> and a single parameter of type <code>Vector[T]</code>.The idea to constrain a parameter to types that are members of a specific type class is realized by means of the <code>implicit</code> second parameter list. What does this mean? Basically, that a value of type <code>NumberLike[T]</code> must be implicitly available in the current scope. This is the case if an implicit value has been declared and made available in the current scope, very often by importing the package or object in which that implicit value is defined.If and only if no other implicit value can be found, the compiler will look in the companion object of the type of the implicit parameter. Hence, as a library designer, putting your default type class implementations in the companion object of your type class trait means that users of your library can easily override these implementations with their own ones, which is exactly what you want. Users can also pass in an explicit value for an implicit parameter to override the implicit values that are in scope.Let’s see if the default type class implementations can be resolved:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 </pre></td><td class="code"><pre><code class="scala">val numbers = Vector[Double](13, 23.0, 42, 45, 61, 73, 96, 100, 199, 420, 900, 3839) println(Statistics.mean(numbers)) </code></pre></td></tr></table></div></figure>Wonderful! If we try this with a <code>Vector[String]</code>, we get an error at compile time, stating that no implicit value could be found for parameter <code>ev: NumberLike[String]</code>. If you don’t like this error message, you can customize it by annotating your type class trait with the <code>@implicitNotFound</code> annotation:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 </pre></td><td class="code"><pre><code class="scala">object Math { import annotation.implicitNotFound @implicitNotFound("No member of type class NumberLike in scope for ${T}") trait NumberLike[T] { def plus(x: T, y: T): T def divide(x: T, y: Int): T def minus(x: T, y: T): T } } </code></pre></td></tr></table></div></figure><h4>Context bounds</h4>A second, implicit parameter list on all methods that expect a member of a type class can be a little verbose. As a shortcut for implicit parameters with only one type parameter, Scala provides so-called context bounds. To show how those are used, we are going to implement our other statistics methods using those instead:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 </pre></td><td class="code"><pre><code class="scala">object Statistics { import Math.NumberLike def mean[T](xs: Vector[T])(implicit ev: NumberLike[T]): T = ev.divide(xs.reduce(ev.plus(_, _)), xs.size) def median[T : NumberLike](xs: Vector[T]): T = xs(xs.size / 2) def quartiles[T: NumberLike](xs: Vector[T]): (T, T, T) = (xs(xs.size / 4), median(xs), xs(xs.size / 4 * 3)) def iqr[T: NumberLike](xs: Vector[T]): T = quartiles(xs) match { case (lowerQuartile, _, upperQuartile) => implicitly[NumberLike[T]].minus(upperQuartile, lowerQuartile) } } </code></pre></td></tr></table></div></figure>A context bound <code>T : NumberLike</code> means that an implicit value of type <code>NumberLike[T]</code> must be available, and so is really equivalent to having a second implicit parameter list with a <code>NumberLike[T]</code> in it. If you want to access that implicitly available value, however, you need to call the <code>implicitly</code> method, as we do in the <code>iqr</code> method. If your type class requires more than one type parameter, you cannot use the context bound syntax.<h3>Custom type class members</h3>As a user of a library that makes use of type classes, you will sooner or later have types that you want to make members of those type classes. For instance, we might want to use the statistics library for instances of the Joda Time <code>Duration</code> type. To do that, we need Joda Time on our classpath, of course:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 </pre></td><td class="code"><pre><code class="scala">libraryDependencies += "joda-time" % "joda-time" % "2.1" libraryDependencies += "org.joda" % "joda-convert" % "1.3" </code></pre></td></tr></table></div></figure>Now we just have to create an implicitly available implementation of <code>NumberLike</code> (please make sure you have Joda Time on your classpath when trying this out):<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 </pre></td><td class="code"><pre><code class="scala">object JodaImplicits { import Math.NumberLike import org.joda.time.Duration implicit object NumberLikeDuration extends NumberLike[Duration] { def plus(x: Duration, y: Duration): Duration = x.plus(y) def divide(x: Duration, y: Int): Duration = Duration.millis(x.getMillis / y) def minus(x: Duration, y: Duration): Duration = x.minus(y) } } </code></pre></td></tr></table></div></figure>If we import the package or object containing this <code>NumberLike</code> implementation, we can now compute the mean value for a bunch of durations:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 </pre></td><td class="code"><pre><code class="scala">import Statistics._ import JodaImplicits._ import org.joda.time.Duration._ val durations = Vector(standardSeconds(20), standardSeconds(57), standardMinutes(2), standardMinutes(17), standardMinutes(30), standardMinutes(58), standardHours(2), standardHours(5), standardHours(8), standardHours(17), standardDays(1), standardDays(4)) println(mean(durations).getStandardHours) </code></pre></td></tr></table></div></figure><h2>Use cases</h2>Our <code>NumberLike</code> type class was a nice exercise, but Scala already ships with the <code>Numeric</code> type class, which allows you to call methods like <code>sum</code> or <code>product</code> on collections for whose type <code>T</code> a <code>Numeric[T]</code> is available. Another type class in the standard library that you will use a lot is <code>Ordering</code>, which allows you to provide an implicit ordering for your own types, available to the <code>sort</code> method on Scala’s collections.There are more type classes in the standard library, but not all of them are ones you have to deal with on a regular basis as a Scala developer.A very common use case in third-party libraries is that of object serialization and deserialization, most notably to and from JSON. By making your classes members of an appropriate formatter type class, you can customize the way your classes are serialized to JSON, XML or whatever format is currently the new black.Mapping between Scala types and ones supported by your database driver is also commonly made customizable and extensible via type classes.<h2>Summary</h2>Once you start to do some serious work with Scala, you will inevitably stumble upon type classes. I hope that after reading this article, you are prepared to take advantage of this powerful technique.Scala type classes allow you to develop your Scala code in such a way that it’s open for retroactive extension while retaining as much concrete type information as possible. In contrast to approaches from other languages, they give developers full control, as default type class implementations can be overridden without much hassle, and type classes implementations are not made available in the global namespace.You will see that this technique is especially useful when writing libraries intended to be used by others, but type classes also have their use in application code to decrease coupling between modules.</div> Mon, 02 Dec 2013 14:00:11 -0000 The Neophyte's Guide to Scala Part 13: Path-dependent types - Daniel Westheide http://danielwestheide.com/blog/2013/02/13/the-neophytes-guide-to-scala-part-13-path-dependent-types.html <div class="entry-content">In last week’s article, I <a href="http://danielwestheide.com/blog/2013/02/06/the-neophytes-guide-to-scala-part-12-type-classes.html">introduced you to the idea of type classes</a> – a pattern that allows you to design your programs to be open for extension without giving up important information about concrete types. This week, I’m going to stick with Scala’s type system and talk about one of its features that distinguishes it from most other mainstream programming languages: Scala’s form of dependent types, in particular path-dependent types and dependent method types.One of the most widely used arguments against static typing is that “the compiler is just in the way” and that in the end, it’s all only data, so why care about building up a complex hierarchy of types?In the end, having static types is all about preventing bugs by allowing the über-smart compiler to humiliate you on a regular basis, making sure you’re doing the right thing before it’s too late.Using path-dependent types is one powerful way to help the compiler prevent you from introducing bugs, as it places logic that is usually only available at runtime into types.Sometimes, accidentally introducing path-dependent types can lead to frustation, though, especially if you have never heard of them. Hence, it’s definitely a good idea to get familiar with them, whether you decide to put them to use or not.<h2>The problem</h2>I will start by presenting a problem that path-dependent types can help us solving: In the realm of fan fiction, the most atrocious things happen – usually, the involved characters will end up making out with each other, regardless how inappropriate it is. There is even crossover fan fiction, in which two characters from different franchises are making out with each other.However, elitist fan fiction writers look down on this. Surely there is a way to prevent such wrongdoing! Here is a first version of our domain model:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 </pre></td><td class="code"><pre><code class="scala">object Franchise { case class Character(name: String) } class Franchise(name: String) { import Franchise.Character def createFanFiction( lovestruck: Character, objectOfDesire: Character): (Character, Character) = (lovestruck, objectOfDesire) } </code></pre></td></tr></table></div></figure>Characters are represented by instances of the <code>Character</code> case class, and the <code>Franchise</code> class has a method to create a new piece of fan fiction about two characters. Let’s create two franchises and some characters:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 </pre></td><td class="code"><pre><code class="scala">val starTrek = new Franchise("Star Trek") val starWars = new Franchise("Star Wars") val quark = Franchise.Character("Quark") val jadzia = Franchise.Character("Jadzia Dax") val luke = Franchise.Character("Luke Skywalker") val yoda = Franchise.Character("Yoda") </code></pre></td></tr></table></div></figure>Unfortunately, at the moment we are unable to prevent bad things from happening:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 </pre></td><td class="code"><pre><code class="scala">starTrek.createFanFiction(lovestruck = jadzia, objectOfDesire = luke) </code></pre></td></tr></table></div></figure>Horrors of horrors! Someone has created a piece of fan fiction in which Jadzia Dax is making out with Luke Skywalker. Preposterous! Clearly, we should not allow this. Your first intuition might be to somehow check at runtime that two characters making out are from the same franchise. For example, we could change the model like so:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 </pre></td><td class="code"><pre><code class="scala">object Franchise { case class Character(name: String, franchise: Franchise) } class Franchise(name: String) { import Franchise.Character def createFanFiction( lovestruck: Character, objectOfDesire: Character): (Character, Character) = { require(lovestruck.franchise == objectOfDesire.franchise) (lovestruck, objectOfDesire) } } </code></pre></td></tr></table></div></figure>Now, the <code>Character</code> instances have a reference to their <code>Franchise</code>, and trying to create a fan fiction with characters from different franchises will lead to an <code>IllegalArgumentException</code> (feel free to try this out in a REPL).<h2>Safer fiction with path-dependent types</h2>This is pretty good, isn’t it? It’s the kind of fail-fast behaviour we have been indoctrinated with for years. However, with Scala, we can do better. There is a way to fail even faster – not at runtime, but at compile time. To achieve that, we need to encode the connection between a <code>Character</code> and its <code>Franchise</code> at the type level.Luckily, the way Scala’s nested types work allow us to do that. In Scala, a nested type is bound to a specific instance of the outer type, not to the outer type itself. This means that if you try to use an instance of the inner type outside of the instance of the enclosing type, you will face a compile error:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 </pre></td><td class="code"><pre><code class="scala">class A { class B var b: Option[B] = None } val a1 = new A val a2 = new A val b1 = new a1.B val b2 = new a2.B a1.b = Some(b1) a2.b = Some(b1) // does not compile </code></pre></td></tr></table></div></figure>You cannot simply assign an instance of the <code>B</code> that is bound to <code>a2</code> to the field on <code>a1</code> – the one is an <code>a2.B</code>, the other expects an <code>a1.B</code>. The dot syntax represents the path to the type, going along concrete instances of other types. Hence the name, path-dependent types.We can put these to use in order to prevent characters from different franchises making out with each other:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 </pre></td><td class="code"><pre><code class="scala">class Franchise(name: String) { case class Character(name: String) def createFanFictionWith( lovestruck: Character, objectOfDesire: Character): (Character, Character) = (lovestruck, objectOfDesire) } </code></pre></td></tr></table></div></figure>Now, the type <code>Character</code> is nested in the type <code>Franchise</code>, which means that it is dependent on a specific enclosing instance of the <code>Franchise</code> type.Let’s create our example franchises and characters again:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 </pre></td><td class="code"><pre><code class="scala">val starTrek = new Franchise("Star Trek") val starWars = new Franchise("Star Wars") val quark = starTrek.Character("Quark") val jadzia = starTrek.Character("Jadzia Dax") val luke = starWars.Character("Luke Skywalker") val yoda = starWars.Character("Yoda") </code></pre></td></tr></table></div></figure>You can already see in how our <code>Character</code> instances are created that their types are bound to a specific franchise. Let’s see what happens if we try to put some of these characters together:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 </pre></td><td class="code"><pre><code class="scala">starTrek.createFanFictionWith(lovestruck = quark, objectOfDesire = jadzia) starWars.createFanFictionWith(lovestruck = luke, objectOfDesire = yoda) </code></pre></td></tr></table></div></figure>These two compile, as expected. They are tasteless, but what can we do?Now, let’s see what happens if we try to create some fan fiction about Jadzia Dax and Luke Skywalker:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 </pre></td><td class="code"><pre><code class="scala">starTrek.createFanFictionWith(lovestruck = jadzia, objectOfDesire = luke) </code></pre></td></tr></table></div></figure>Et voilà: The thing that should not be does not even compile! The compiler complains about a type mismatch:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 </pre></td><td class="code"><pre><code class="">found : starWars.Character required: starTrek.Character starTrek.createFanFictionWith(lovestruck = jadzia, objectOfDesire = luke) ^</code></pre></td></tr></table></div></figure>This technique also works if our method is not defined on the <code>Franchise</code> class, but in some other module. In this case, we can make use of dependent method types, where the type of one parameter depends on a previous parameter:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 </pre></td><td class="code"><pre><code class="scala">def createFanFiction(f: Franchise)(lovestruck: f.Character, objectOfDesire: f.Character) = (lovestruck, objectOfDesire) </code></pre></td></tr></table></div></figure>As you can see, the type of the <code>lovestruck</code> and <code>objectOfDesire</code> parameters depends on the <code>Franchise</code> instance passed to the method. Note that this only works if the instance on which other types depend is in its own parameter list.<h2>Abstract type members</h2>Often, dependent method types are used in conjunction with abstract type members. Suppose we want to develop a hipsterrific key-value store. It will only support setting and getting the value for a key, but in a typesafe manner. Here is our oversimplified implementation:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 </pre></td><td class="code"><pre><code class="scala">object AwesomeDB { abstract class Key(name: String) { type Value } } import AwesomeDB.Key class AwesomeDB { import collection.mutable.Map val data = Map.empty[Key, Any] def get(key: Key): Option[key.Value] = data.get(key).asInstanceOf[Option[key.Value]] def set(key: Key)(value: key.Value): Unit = data.update(key, value) } </code></pre></td></tr></table></div></figure>We have defined a class <code>Key</code> with an abstract type member <code>Value</code>. The methods on <code>AwesomeDB</code> refer to that type without ever knowing or caring about the specific manifestation of this abstract type.We can now define some concrete keys that we want to use:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 </pre></td><td class="code"><pre><code class="scala">trait IntValued extends Key { type Value = Int } trait StringValued extends Key { type Value = String } object Keys { val foo = new Key("foo") with IntValued val bar = new Key("bar") with StringValued } </code></pre></td></tr></table></div></figure>Now we can set and get key/value pairs in a typesafe manner:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 </pre></td><td class="code"><pre><code class="scala">val dataStore = new AwesomeDB dataStore.set(Keys.foo)(23) val i: Option[Int] = dataStore.get(Keys.foo) dataStore.set(Keys.foo)("23") // does not compile </code></pre></td></tr></table></div></figure><h2>Path-dependent types in practice</h2>While path-dependent types are not necessarily omnipresent in your typical Scala code, they do have a lot of practical value beyond modelling the domain of fan fiction.One of the most widespread uses is probably seen in combination with the cake pattern, which is a technique for composing your components and managing their dependencies, relying solely on features of the language. See the excellent articles <a href="http://debasishg.blogspot.ie/2013/02/modular-abstractions-in-scala-with.html">by Debasish Ghosh</a> and <a href="http://precog.com/blog/Existential-Types-FTW/">Precog’s Daniel Spiewak</a> to learn more about both the cake pattern and how it can be improved by incorporating path-dependent types.In general, whenever you want to make sure that objects created or managed by a specific instance of another type cannot accidentally or purposely be interchanged or mixed, path-dependent types are the way to go.Path-dependent types and dependent method types play a crucial role for attempts to encode information into types that is typically only known at runtime, for instance heterogenous lists, type-level representations of natural numbers and collections that carry their size in their type. Miles Sabin is exploring the limits of Scala’s type system in this respect in his excellent library <a href="https://github.com/milessabin/shapeless">Shapeless</a>.</div> Thu, 05 Dec 2013 14:00:12 -0000 The Neophyte's Guide to Scala Part 14: The Actor approach to concurrency - Daniel Westheide http://danielwestheide.com/blog/2013/02/27/the-neophytes-guide-to-scala-part-14-the-actor-approach-to-concurrency.html <div class="entry-content">After several articles about how you can leverage the Scala type system to achieve a great amount of both flexibility and compile-time safety, we are now shifting back to a topic that we already tackled previously in this series: Scala’s take on concurrency.In <a href="http://danielwestheide.com/blog/2013/01/09/the-neophytes-guide-to-scala-part-8-welcome-to-the-future.html">these</a> earlier <a href="http://danielwestheide.com/blog/2013/01/16/the-neophytes-guide-to-scala-part-9-promises-and-futures-in-practice.html">articles</a>, you learned about an approach that allows you to work asynchronously by making use of composable Futures.This approach is a very good fit for numerous problems. However, it’s not the only one Scala has to offer. A second cornerstone of Scala concurrency is the Actor model. It provides an approach to concurrency that is entirely based on passing messages between processes.Actors are not a new idea – the most prominent implementation of this model can be found in Erlang. The Scala core library has had its own actors library for a long time, but it faces the destiny of deprecation in the coming Scala version 2.11, when it will ultimately be replaced by the actors implementation provided by the <a href="http://akka.io">Akka</a> toolkit, which has been a de-facto standard for actor-based development with Scala for quite a while.In this article, you will be introduced to the rationale behind Akka’s actor model and learn the basics of coding within this paradigm using the Akka toolkit. It is by no means an in-depth discussion of everything you need to know about Akka actors, and in that, it differs from most of the previous articles in this series. Rather, the intention is to familiarize you with the Akka mindset and serve as an initial spark to get you excited about it.<h2>The problems with shared mutable state</h2>The predominant approach to concurrency today is that of shared mutable state – a large number of stateful objects whose state can be changed by multiple parts of your application, each running in their own thread. Typically, the code is interspersed with read and write locks, to make sure that the state can only be changed in a controlled way and prevent multiple threads from mutating it simultaneously. At the same time, we are trying hard not to lock too big a block of code, as this can drastically slow down the application.More often than not, code like this has originally been written without having concurrency in mind at all – only to be made fit for a multi-threaded world once the need arose. While writing software without the need for concurrency like this leads to very straightforward code, adapting it to the needs of a concurrent world leads to code that is really, really difficult to read and understand.The core problem is that low-level synchronization constructs like locks and threads are very hard to reason about. As a consequence, it’s very hard to get it right: If you can’t easily reason about what’s going on, you can be sure that nasty bugs will ensue, from race conditions to deadlocks or just strange behaviour – maybe you’ll only notice after some months, long after your code has been deployed to your production servers.Also, working with these low-level constructs makes it a real challenge to achieve an acceptable performance.<h2>The Actor model</h2>The Actor programming model is aimed at avoiding all the problems described above, allowing you to write highly performant concurrent code that is easy to reason about. Unlike the widely used approach of shared mutable state, it requires you to design and write your application from the ground up with concurrency in mind – it’s not really possible to add support for it later on.The idea is that your application consists of lots of light-weight entities called actors. Each of these actors is responsible for only a very small task, and is thus easy to reason about. A more complex business logic arises out of the interaction between several actors, delegating tasks to others or passing messages to collaborators for other reasons.<h3>The Actor System</h3>Actors are pitiful creatures: They cannot live on their own. Rather, each and every actor in Akka resides in and is created by an actor system. Aside from allowing you to create and find actors, an <code>ActorSystem</code> provides for a whole bunch of additional functionality, none of which shall concern us right now.In order to try out the example code, please add the following resolver and dependency to your SBT-based Scala 2.10 project first:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 </pre></td><td class="code"><pre><code class="scala">resolvers += "Typesafe Releases" at "http://repo.typesafe.com/typesafe/releases" libraryDependencies += "com.typesafe.akka" %% "akka-actor" % "2.2.3" </code></pre></td></tr></table></div></figure>Now, let’s create an <code>ActorSystem</code>. We’ll need it as an environment for our actors:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 </pre></td><td class="code"><pre><code class="scala">import akka.actor.ActorSystem object Barista extends App { val system = ActorSystem("Barista") system.shutdown() } </code></pre></td></tr></table></div></figure>We created a new instance of <code>ActorSystem</code> and gave it the name <code>"Barista"</code> – we are returning to the domain of coffee, which should be familiar from the article on <a href="http://danielwestheide.com/blog/2013/01/09/the-neophytes-guide-to-scala-part-8-welcome-to-the-future.html">composable futures</a>.Finally, we are good citizens and shut down our actor system once we no longer need it.<h3>Defining an actor</h3>Whether your application consists of a few dozen or a few million actors totally depends on your use case, but Akka is absolutely okay with a few million. You might be baffled by this insanely high number. It’s important to understand that there is not a one-to-one relationship between an actor and a thread. You would soon run out of memory if that were the case. Rather, due to the non-blocking nature of actors, one thread can execute many actors – switching between them depending on which of them has messages to be processed.To understand what is actually happening, let’s first create a very simple actor, a <code>Barista</code> that can receive orders but doesn’t really do anything apart from printing messages to the console:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 </pre></td><td class="code"><pre><code class="scala">sealed trait CoffeeRequest case object CappuccinoRequest extends CoffeeRequest case object EspressoRequest extends CoffeeRequest import akka.actor.Actor class Barista extends Actor { def receive = { case CappuccinoRequest => println("I have to prepare a cappuccino!") case EspressoRequest => println("Let's prepare an espresso.") } } </code></pre></td></tr></table></div></figure>First, we define the types of messages that our actor understands. Typically, case classes are used for messages sent between actors if you need to pass along any parameters. If all the actor needs is an unparameterized message, this message is typically represented as a case object – which is exactly what we are doing here.In any case, it’s crucial that your messages are immutable, or else bad things will happen.Next, let’s have a look at our class <code>Barista</code>, which is the actual actor, extending the aptly named <code>Actor</code> trait. Said trait defines a method <code>receive</code> which returns a value of type <code>Receive</code>. The latter is really only a type alias for <code>PartialFunction[Any, Unit]</code>.<h3>Processing messages</h3>So what’s the meaning of this <code>receive</code> method? The return type, <code>PartialFunction[Any, Unit]</code> may seem strange to you in more than one respect.In a nutshell, the partial function returned by the <code>receive</code> method is responsible for processing your messages. Whenever another part of your software – be it another actor or not – sends your actor a message, Akka will eventually let it process this message by calling the partial function returned by your actor’s <code>receive</code> method, passing it the message as an argument.<h4>Side-effecting</h4>When processing a message, an actor can do whatever you want it to, apart from returning a value.Wat!?As the return type of <code>Unit</code> suggests, your partial function is side-effecting. This might come as a bit of a shock to you after we emphasized the usage of pure functions all the time. For a concurrent programming model, this actually makes a lot of sense. Actors are where your state is located, and having some clearly defined places where side-effects will occur in a controllable manner is totally fine – each message your actor receives is processed in isolation, one after another, so there is no need to reason about synchronization or locks.<h4>Untyped</h4>But… this partial function is not only side-effecting, it’s also as untyped as you can get in Scala, expecting an argument of type <code>Any</code>. Why is that, when we have such a powerful type system at our fingertips?This has a lot to do with some important design choices in Akka that allow you to do things like forwarding messages to other actors, installing load balancing or proxying actors without the sender having to know anything about them and so on.In practice, this is usually not a problem. With the messages themselves being strongly typed, you typically use pattern matching for processing those types of messages you are interested in, just as we did in our tiny example above.Sometimes though, the weakly typed actors can indeed lead to nasty bugs the compiler can’t catch for you. If you have grown to love the benefits of a strong type system and think you don’t want to go away from that at any costs for some parts of your application, you may want to look at Akka’s new experimental <a href="http://doc.akka.io/docs/akka/snapshot/scala/typed-channels.html">Typed Channels</a> feature.<h4>Asynchronous and non-blocking</h4>I wrote above that Akka would let your actor eventually process a message sent to it. This is important to keep in mind: Sending a message and processing it is done in an asynchronous and non-blocking fashion. The sender will not be blocked until the message has been processed by the receiver. Instead, they can immediately continue with their own work. Maybe they expect to get a messsage from your actor in return after a while, or maybe they are not interested in hearing back from your actor at all.What really happens when some component sends a message to an actor is that this message is delivered to the actor’s mailbox, which is basically a queue. Placing a message in an actor’s mailbox is a non-blocking operation, i.e. the sender doesn’t have to wait until the message is actually enqueued in the recipient’s mailbox.The dispatcher will notice the arrival of a new message in an actor’s mailbox, again asynchronously. If the actor is not already processing a previous message, it is now allocated to one of the threads available in the execution context. Once the actor is done processing any previous messages, the dispatcher sends it the next message from its mailbox for processing.The actor blocks the thread to which it is allocated for as long as it takes to process the message. While this doesn’t block the sender of the message, it means that lengthy operations degrade overall performance, as all the other actors have to be scheduled for processing messages on one of the remaining threads.Hence, a core principle to follow for your <code>Receive</code> partial functions is to spend as little time inside them as possible. Most importantly, avoid calling blocking code inside your message processing code, if possible at all.Of course, this is something you can’t prevent doing completely – the majority of database drivers nowadays is still blocking, and you will want to be able to persist data or query for it from your actor-based application. There are solutions to this dilemma, but we won’t cover them in this introductory article.<h3>Creating an actor</h3>Defining an actor is all well and good, but how do we actually use our <code>Barista</code> actor in our application? To do that, we have to create a new instance of our <code>Barista</code> actor. You might be tempted to do it the usual way, by calling its constructor like so:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 </pre></td><td class="code"><pre><code class="scala">val barista = new Barista // will throw exception </code></pre></td></tr></table></div></figure>This will not work! Akka will thank you with an <code>ActorInitializationException</code>. The thing is, in order for the whole actor thingie to work properly, your actors need to be managed by the <code>ActorSystem</code> and its components. Hence, you have to ask the actor system for a new instance of your actor:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 </pre></td><td class="code"><pre><code class="scala">import akka.actor.{ActorRef, Props} val barista: ActorRef = system.actorOf(Props[Barista], "Barista") </code></pre></td></tr></table></div></figure>The <code>actorOf</code> method defined on <code>ActorSystem</code> expects a <code>Props</code> instance, which provides a means of configuring newly created actors, and, optionally, a name for your actor instance. We are using the simplest form of creating such a <code>Props</code> instance, providing the <code>apply</code> method of the companion object with a type parameter. Akka will then create a new instance of the actor of the given type by calling its default constructor.Be aware that the type of the object returned by <code>actorOf</code> is not <code>Barista</code>, but <code>ActorRef</code>. Actors never communicate with another directly and hence there are supposed to be no direct references to actor instances. Instead, actors or other components of your application aquire references to the actors they need to send messages to.Thus, an <code>ActorRef</code> acts as some kind of proxy to the actual actor. This is convenient because an <code>ActorRef</code> can be serialized, allowing it to be a proxy for a remote actor on some other machine. For the component aquiring an <code>ActorRef</code>, the location of the actor – local in the same JVM or remote on some other machine – is completely transparent. We call this property location transparency.Please note that <code>ActorRef</code> is not parameterized by type. Any <code>ActorRef</code> can be exchanged for another, allowing you to send arbitrary messages to any <code>ActorRef</code>. This is by design and, as already mentioned above, allows for easily modifying the topology of your actor system wihout having to make any changes to the senders.<h3>Sending messages</h3>Now that we have created an instance of our <code>Barista</code> actor and got an <code>ActorRef</code> linked to it, we can send it a message. This is done by calling the <code>!</code> method on the <code>ActorRef</code>:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 </pre></td><td class="code"><pre><code class="scala">barista ! CappuccinoRequest barista ! EspressoRequest println("I ordered a cappuccino and an espresso") </code></pre></td></tr></table></div></figure>Calling the <code>!</code> is a fire-and-forget operation: You tell the <code>Barista</code> that you want a cappuccino, but you don’t wait for their response. It’s the most common way in Akka for interacting with other actors. By calling this method, you tell Akka to enqueue your message in the recipient’s mailbox. As described above, this doesn’t block, and eventually the recipient actor will process your message.Due to the asynchronous nature, the result of the above code is not deterministic. It might look like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 </pre></td><td class="code"><pre><code class="">I have to prepare a cappuccino! I ordered a cappuccino and an espresso Let's prepare an espresso.</code></pre></td></tr></table></div></figure>Even though we first sent the two messages to the <code>Barista</code> actor’s mailbox, between the processing of the first and second message, our own output is printed to the console.<h3>Answering to messages</h3>Sometimes, being able to tell others what to do just doesn’t cut it. You would like to be able to answer by in turn sending a message to the sender of a message you got – all asynchronously of course.To enable you to do that and lots of other things that are of no concern to us right now, actors have a method called <code>sender</code>, which returns the <code>ActorRef</code> of the sender of the last message, i.e. the one you are currently processing.But how does it know about that sender? The answer can be found in the signature of the <code>!</code> method, which has a second, implicit parameter list:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 </pre></td><td class="code"><pre><code class="scala">def !(message: Any)(implicit sender: ActorRef = Actor.noSender): Unit </code></pre></td></tr></table></div></figure>When called from an actor, its <code>ActorRef</code> is passed on as the implicit <code>sender</code> argument.Let’s change our <code>Barista</code> so that they immediately send a <code>Bill</code> to the sender of a <code>CoffeeRequest</code> before printing their usual output to the console:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 </pre></td><td class="code"><pre><code class="scala">case class Bill(cents: Int) case object ClosingTime class Barista extends Actor { def receive = { case CappuccinoRequest => sender ! Bill(250) println("I have to prepare a cappuccino!") case EspressoRequest => sender ! Bill(200) println("Let's prepare an espresso.") case ClosingTime => context.system.shutdown() } } </code></pre></td></tr></table></div></figure>While we are at it, we are introducing a new message, <code>ClosingTime</code>. The <code>Barista</code> reacts to it by shutting down the actor system, which they, like all actors, can access via their <code>ActorContext</code>.Now, let’s introduce a second actor representing a customer:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 </pre></td><td class="code"><pre><code class="scala">case object CaffeineWithdrawalWarning class Customer(caffeineSource: ActorRef) extends Actor { def receive = { case CaffeineWithdrawalWarning => caffeineSource ! EspressoRequest case Bill(cents) => println(s"I have to pay $cents cents, or else!") } } </code></pre></td></tr></table></div></figure>This actor is a real coffee junkie, so it needs to be able to order new coffee. We pass it an <code>ActorRef</code> in the constructor – for the <code>Customer</code>, this is simply its <code>caffeineSource</code> – it doesn’t know whether this <code>ActorRef</code> points to a <code>Barista</code> or something else. It knows that it can send <code>CoffeeRequest</code> messages to it, and that is all that matters to them.Finally, we need to create these two actors and send the customer a <code>CaffeineWithdrawalWarning</code> to get things rolling:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 </pre></td><td class="code"><pre><code class="scala">val barista = system.actorOf(Props[Barista], "Barista") val customer = system.actorOf(Props(classOf[Customer], barista), "Customer") customer ! CaffeineWithdrawalWarning barista ! ClosingTime </code></pre></td></tr></table></div></figure>Here, for the <code>Customer</code> actor, we are using a different factory method for creating a <code>Props</code> instance: We pass in the type of the actor we want to have instantiated as well as the constructor arguments that actor takes. We need to do this because we want to pass the <code>ActorRef</code> of our <code>Barista</code> actor to the constructor of the <code>Customer</code> actor.Sending the <code>CaffeineWithdrawalWarning</code> to the customer makes it send an <code>EspressoRequest</code> to the barista who will then send a <code>Bill</code> back to the customer. The output of this may look like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 </pre></td><td class="code"><pre><code class="">Let's prepare an espresso. I have to pay 200 cents, or else!</code></pre></td></tr></table></div></figure>First, while processing the <code>EspressoRequest</code> message, the <code>Barista</code> sends a message to the sender of that message, the <code>Customer</code> actor. However, this operation doesn’t block until the latter processes it. The <code>Barista</code> actor can continue processing the <code>EspressoRequest</code> immediately, and does this by printing to the console. Shortly after, the <code>Customer</code> starts to process the <code>Bill</code> message and in turn prints to the console.<h3>Asking questions</h3>Sometimes, sending an actor a message and expecting a message in return at some later time isn’t an option – the most common place where this is the case is in components that need to interface with actors, but are not actors themselves. Living outside of the actor world, they cannot receive messages.For situations such as these, there is Akka’s ask support, which provides some sort of bridge between actor-based and future-based concurrency. From the client perspective, it works like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 </pre></td><td class="code"><pre><code class="scala">import akka.pattern.ask import akka.util.Timeout import scala.concurrent.duration._ implicit val timeout = Timeout(2.second) implicit val ec = system.dispatcher val f: Future[Any] = barista2 ? CappuccinoRequest f.onSuccess { case Bill(cents) => println(s"Will pay $cents cents for a cappuccino") } </code></pre></td></tr></table></div></figure>First, you need to import support for the ask syntax and create an implicit timeout for the <code>Future</code> returned by the <code>?</code> method. Also, the <code>Future</code> needs an <code>ExecutionContext</code>. Here, we simply use the default dispatcher of our <code>ActorSystem</code>, which is conveniently also an <code>ExecutionContext</code>.As you can see, the returned future is untyped – it’s a <code>Future[Any]</code>. This shouldn’t come as a surprise, since it’s really a received message from an actor, and those are untyped, too.For the actor that is being asked, this is actually the same as sending some message to the sender of a processed message. This is why asking our <code>Barista</code> works out of the box without having to change anything in our <code>Barista</code> actor.Once the actor being asked sends a message to the sender, the <code>Promise</code> belonging to the returned <code>Future</code> is completed.Generally, telling is preferable to asking, because it’s more resource-sparing. Akka is not for polite people! However, there are situations where you really need to ask, and then it’s perfectly fine to do so.<h3>Stateful actors</h3>Each actor may maintain an internal state, but that’s not strictly necessary. Sometimes, a large part of the overall application state consists of the information carried by the immutable messages passed between actors.An actor only ever processes one message at a time. While doing so, it may modify its internal state. This means that there is some kind of mutable state in an actor, but since each message is processed in isolation, there is no way the internal state of our actor can get messed up due to concurrency problems.To illustrate, let’s turn our stateless <code>Barista</code> into an actor carrying state, by simply counting the number of orders:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 </pre></td><td class="code"><pre><code class="scala">class Barista extends Actor { var cappuccinoCount = 0 var espressoCount = 0 def receive = { case CappuccinoRequest => sender ! Bill(250) cappuccinoCount += 1 println(s"I have to prepare cappuccino #$cappuccinoCount") case EspressoRequest => sender ! Bill(200) espressoCount += 1 println(s"Let's prepare espresso #$espressoCount.") case ClosingTime => context.system.shutdown() } } </code></pre></td></tr></table></div></figure>We introduced two <code>vars</code>, <code>cappuccinoCount</code> and <code>espressoCount</code> that are incremented with each respective order. This is actually the first time in this series that we have used a <code>var</code>. While to be avoided in functional programming, they are really the only way to allow your actors to carry state. Since each message is processed in isolation, our above code is similar to using <code>AtomicInteger</code> values in a non-actor environment.<h2>Conclusion</h2>And here ends our introduction to the actor programming model for concurrency and how to work within this paradigm using Akka. While we have really only scratched the surface and have ignored some important concepts of Akka, I hope to have given enough of an insight into this approach to concurrency to give you a basic understanding and get you interested in learning more.In the coming articles, I will elaborate our little example, adding some meaningful behaviour to it while introducing more of the ideas behind Akka actors, among them the question of how errors are handled in an actor system.P.S. Please note that starting with this article I have switched to a biweekly schedule for the remaining parts of this series.</div> Mon, 09 Dec 2013 14:00:21 -0000 The Neophyte's Guide to Scala Part 15: Dealing with Failure in Actor Systems - Daniel Westheide http://danielwestheide.com/blog/2013/03/20/the-neophytes-guide-to-scala-part-15-dealing-with-failure-in-actor-systems.html <div class="entry-content">In the previous part of this series, I introduced you to the second cornerstone of Scala concurrency: The actor model, which complements the model based on composable futures backed by promises. You learnt how to define and create actors, how to send messages to them and how an actor processes these messages, possibly modifying its internal state as a result or asynchronously sending a response message to the sender.While that was hopefully enough to get you interested in the actor model for concurrency, I left out some crucial concepts you will want to know about before starting to develop actor-based applications that consist of more than a simple echo actor.The actor model is meant to help you achieve a high level of fault tolerance. In this article, we are going to have a look at how to deal with failure in an actor-based application, which is fundamentally different from error handling in a traditional layered server architecture.The way you deal with failure is closely linked to some core Akka concepts and to some of the elements an actor system in Akka consists of. Hence, this article will also serve as a guide to those ideas and components.<h2>Actor hierarchies</h2>Before going into what happens when an error occurs in one of your actors, it’s essential to introduce one crucial idea underlying the actor approach to concurrency – an idea that is the very foundation for allowing you to build fault-tolerant concurrent applications: Actors are organized in a hierarchy.So what does this mean? First of all, it means that every single of your actors has got a parent actor, and that each actor can create child actors. Basically, you can think of an actor system as a pyramid of actors. Parent actors watch over their children, just as in real life, taking care that they get back on their feet if they stumble. You will see shortly how exactly this is done.<h3>The guardian actor</h3>In the previous article, we only had two different actors, a <code>Barista</code> actor and a <code>Customer</code> actor. I will not repeat their rather trivial implementations, but focus on how we created instances of these actor types:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 </pre></td><td class="code"><pre><code class="scala">import akka.actor.ActorSystem val system = ActorSystem("Coffeehouse") val barista = system.actorOf(Props[Barista], "Barista") val customer = system.actorOf(Props(classOf[Customer], barista), "Customer") </code></pre></td></tr></table></div></figure>As you can see, we create these two actors by calling the <code>actorOf</code> method defined on the <code>ActorSystem</code> type.So what is the parent of these two actors? Is it the actor system? Not quite, but close. The actor system is not an actor itself, but it has got a so-called guardian actor that serves as the parent of all root-level user actors, i.e. actors we create by calling <code>actorOf</code> on our actor system.There shouldn’t be a whole lot of actors in your system that are children of the guardian actor. It really makes more sense to have only a few top-level actors, each of them delegating most of the work to their children.<h3>Actor paths</h3>The hierarchical structure of an actor system becomes apparent when looking at the actor paths of the actors you create. These are basically URLs by which actors can be addressed. You can get an actor’s path by calling <code>path</code> on its <code>ActorRef</code>:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 </pre></td><td class="code"><pre><code class="scala">barista.path // => akka.actor.ActorPath = akka://Coffeehouse/user/Barista customer.path // => akka.actor.ActorPath = akka://Coffeehouse/user/Customer </code></pre></td></tr></table></div></figure>The <code>akka</code> protocol is followed by the name of our actor system, the name of the user guardian actor and, finally, the name we gave our actor when calling <code>actorOf</code> on the system. In the case of remote actors, running on different machines, you would additionally see a host and a port.Actor paths can be used to look up another actor. For example, instead of requiring the barista reference in its constructor, the <code>Customer</code> actor could call the <code>actorSelection</code> method on its <code>ActorContext</code>, passing in a relative path to retrieve a reference to the barista:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 </pre></td><td class="code"><pre><code class="scala">context.actorSelection("../Barista") </code></pre></td></tr></table></div></figure>However, while being able to look up an actor by its path can sometimes come in handy, it’s often a much better idea to pass in dependencies in the constructor, just as we did before. Too much intimate knowledge about where your dependencies are located in the actor system makes your system more susceptible to bugs, and it will be difficult to refactor later on.<h3>An example hierarchy</h3>To illustrate how parents watch over their children and what this has got to do with keeping your system fault-tolerant, I’m going to stick to the domain of the coffeehouse. Let’s give the <code>Barista</code> actor a child actor to which it can delegate some of the work involved in running a coffeehouse.If we really were to model the work of a barista, we were likely giving them a bunch of child actors for all the various subtasks. But to keep this article focused, we have to be a little simplistic with our example.Let’s assume that the barista has got a register. It processes transactions, printing appropriate receipts and incrementing the day’s sales so far. Here is a first version of it:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 </pre></td><td class="code"><pre><code class="scala">import akka.actor._ object Register { sealed trait Article case object Espresso extends Article case object Cappuccino extends Article case class Transaction(article: Article) } class Register extends Actor { import Register._ import Barista._ var revenue = 0 val prices = Map[Article, Int](Espresso -> 150, Cappuccino -> 250) def receive = { case Transaction(article) => val price = prices(article) sender ! createReceipt(price) revenue += price } def createReceipt(price: Int): Receipt = Receipt(price) } </code></pre></td></tr></table></div></figure>It contains an immutable map of the prices for each article, and an integer variable representing the revenue. Whenever it receives a <code>Transaction</code> message, it increments the revenue accordingly and returns a printed receipt to the sender.The <code>Register</code> actor, as already mentioned, is supposed to be a child actor of the <code>Barista</code> actor, which means that we will not create it from our actor system, but from within our <code>Barista</code> actor. The initial version of our actor-come-parent looks like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 </pre></td><td class="code"><pre><code class="scala">object Barista { case object EspressoRequest case object ClosingTime case class EspressoCup(state: EspressoCup.State) object EspressoCup { sealed trait State case object Clean extends State case object Filled extends State case object Dirty extends State } case class Receipt(amount: Int) } class Barista extends Actor { import Barista._ import Register._ import EspressoCup._ import context.dispatcher import akka.util.Timeout import akka.pattern.ask import akka.pattern.pipe import concurrent.duration._ implicit val timeout = Timeout(4.seconds) val register = context.actorOf(Props[Register], "Register") def receive = { case EspressoRequest => val receipt = register ? Transaction(Espresso) receipt.map((EspressoCup(Filled), _)).pipeTo(sender) case ClosingTime => context.stop(self) } } </code></pre></td></tr></table></div></figure>First off, we define the message types that our <code>Barista</code> actor is able to deal with. An <code>EspressoCup</code> can have one out of a fixed set of states, which we ensure by using a sealed trait.The more interesting part is to be found in the implementation of the <code>Barista</code> class. The <code>dispatcher</code>, <code>ask</code>, and <code>pipe</code> imports as well as the implicit timeout are required because we make use of Akka’s ask syntax and futures in our <code>Receive</code> partial function: When we receive an <code>EspressoRequest</code>, we ask the <code>Register</code> actor for a <code>Receipt</code> for our <code>Transaction</code>. This is then combined with a filled espresso cup and piped to the sender, which will thus receive a tuple of type <code>(EspressoCup, Receipt)</code>. This kind of delegating subtasks to child actors and then aggregating or amending their work is typical for actor-based applications.Also, note how we create our child actor by calling <code>actorOf</code> on our <code>ActorContext</code> instead of the <code>ActorSystem</code>. By doing so, the actor we create becomes a child actor of the one who called this method, instead of a top-level actor whose parent is the guardian actor.Finally, here is our <code>Customer</code> actor, which, like the <code>Barista</code> actor, will sit at the top level, just below the guardian actor:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 </pre></td><td class="code"><pre><code class="scala">object Customer { case object CaffeineWithdrawalWarning } class Customer(coffeeSource: ActorRef) extends Actor with ActorLogging { import Customer._ import Barista._ import EspressoCup._ def receive = { case CaffeineWithdrawalWarning => coffeeSource ! EspressoRequest case (EspressoCup(Filled), Receipt(amount)) => log.info(s"yay, caffeine for ${self}!") } } </code></pre></td></tr></table></div></figure>It is not terribly interesting for our tutorial, which focuses more on the <code>Barista</code> actor hierarchy. What’s new is the use of the <code>ActorLogging</code> trait, which allows us to write to the log instead of printing to the console.Now, if we create our actor system and populate it with a <code>Barista</code> and two <code>Customer</code> actors, we can happily feed our two under-caffeinated addicts with a shot of black gold:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 </pre></td><td class="code"><pre><code class="scala">import Customer._ val system = ActorSystem("Coffeehouse") val barista = system.actorOf(Props[Barista], "Barista") val customerJohnny = system.actorOf(Props(classOf[Customer], barista), "Johnny") val customerAlina = system.actorOf(Props(classOf[Customer], barista), "Alina") customerJohnny ! CaffeineWithdrawalWarning customerAlina ! CaffeineWithdrawalWarning </code></pre></td></tr></table></div></figure>If you try this out, you should see two log messages from happy customers.<h2>To crash or not to crash?</h2>Of course, what we are really interested in, at least in this article, is not happy customers, but the question of what happens if things go wrong.Our register is a fragile device – its printing functionality is not as reliable as it should be. Every so often, a paper jam causes it to fail. Let’s add a <code>PaperJamException</code> type to the <code>Register</code> companion object:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 </pre></td><td class="code"><pre><code class="scala">class PaperJamException(msg: String) extends Exception(msg) </code></pre></td></tr></table></div></figure>Then, let’s change the <code>createReceipt</code> method in our <code>Register</code> actor accordingly:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 </pre></td><td class="code"><pre><code class="scala">def createReceipt(price: Int): Receipt = { import util.Random if (Random.nextBoolean()) throw new PaperJamException("OMG, not again!") Receipt(price) } </code></pre></td></tr></table></div></figure>Now, when processing a <code>Transaction</code> message, our <code>Register</code> actor will throw a <code>PaperJamException</code> in about half of the cases.What effect does this have on our actor system, or on our whole application? Luckily, Akka is very robust and not affected by exceptions in our code at all. What happens, though, is that the parent of the misbehaving child is notified – remember that parents are watching over their children, and this is the situation where they have to decide what to do.<h3>Supervisor strategies</h3>The whole act of being notified about exceptions in child actors, however, is not handled by the parent actor’s <code>Receive</code> partial function, as that would confound the parent actor’s own behaviour with the logic for dealing with failure in its children. Instead, the two responsibilities are clearly separated.Each actor defines its own supervisor strategy, which tells Akka how to deal with certain types of errors occurring in your children.There are basically two different types of supervisor strategy, the <code>OneForOneStrategy</code> and the <code>AllForOneStrategy</code>. Choosing the former means that the way you want to deal with an error in one of your children will only affect the child actor from which the error originated, whereas the latter will affect all of your child actors. Which of those strategies is best depends a lot on your individual application.Regardless of which type of <code>SupervisorStrategy</code> you choose for your actor, you will have to specify a <code>Decider</code>, which is a <code>PartialFunction[Throwable, Directive]</code> – this allows you to match against certain subtypes of <code>Throwable</code> and decide for each of them what’s supposed to happen to your problematic child actor (or all your child actors, if you chose the all-for-one strategy).<h3>Directives</h3>Here is a list of the available directives:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 </pre></td><td class="code"><pre><code class="scala">sealed trait Directive case object Resume extends Directive case object Restart extends Directive case object Stop extends Directive case object Escalate extends Directive </code></pre></td></tr></table></div></figure><ul><li>Resume: If you choose to <code>Resume</code>, this probably means that you think of your child actor as a little bit of a drama queen. You decide that the exception was not so exceptional after all – the child actor or actors will simply resume processing messages as if nothing extraordinary had happened.</li><li>Restart: The <code>Restart</code> directive causes Akka to create a new instance of your child actor or actors. The reasoning behind this is that you assume that the internal state of the child/children is corrupted in some way so that it can no longer process any further messages. By restarting the actor, you hope to put it into a clean state again.</li><li>Stop: You effectively kill the actor. It will not be restarted.</li><li>Escalate: If you choose to <code>Escalate</code>, you probably don’t know how to deal with the failure at hand. You delegate the decision about what to do to your own parent actor, hoping they are wiser than you. If an actor escalates, they may very well be restarted themselves by their parent, as the parent will only decide about its own child actors.</li></ul><h3>The default strategy</h3>You don’t have to specify your own supervisor strategy in each and every actor. In fact, we haven’t done that so far. This means that the default supervisor strategy will take effect. It looks like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 </pre></td><td class="code"><pre><code class="scala">final val defaultStrategy: SupervisorStrategy = { def defaultDecider: Decider = { case _: ActorInitializationException ⇒ Stop case _: ActorKilledException ⇒ Stop case _: Exception ⇒ Restart } OneForOneStrategy()(defaultDecider) } </code></pre></td></tr></table></div></figure>This means that for exceptions other than <code>ActorInitializationException</code> or <code>ActorKilledException</code>, the respective child actor in which the exception was thrown will be restarted.Hence, when a <code>PaperJamException</code> occurs in our <code>Register</code> actor, the supervisor strategy of the parent actor (the barista) will cause the <code>Register</code> to be restarted, because we haven’t overridden the default strategy.If you try this out, you will likely see an exception stacktrace in the log, but nothing about the <code>Register</code> actor being restarted.Let’s verify that this is really happening. To do so, however, you will need to learn about the actor lifecycle.<h3>The actor lifecycle</h3>To understand what the directives of a supervisor strategy actually do, it’s crucial to know a little bit about an actor’s lifecycle. Basically, it boils down to this: when created via <code>actorOf</code>, an actor is started. It can then be restarted an arbitrary number of times, in case there is a problem with it. Finally, an actor can be stopped, ultimately leading to its death.There are numerous lifecycle hook methods that an actor implementation can override. It’s also important to know their default implementations. Let’s go through them briefly:<ul><li>preStart: Called when an actor is started, allowing you to do some initialization logic. The default implementation is empty.</li><li>postStop: Empty by default, allowing you to clean up resources. Called after <code>stop</code> has been called for the actor.</li><li>preRestart: Called right before a crashed actor is restarted. By default, it stops all children of that actor and then calls <code>postStop</code> to allow cleaning up of resources.</li><li>postRestart: Called immediately after an actor has been restarted. Simply calls <code>preStart</code> by default.</li></ul>This means that by default, restarting an actor entails a restart of its children. This may be exactly what you want, depending on your specific actor and use case. If it’s not what you want, these hook methods allow you to change that behaviour.Let’s see if our <code>Register</code> gets indeed restarted upon failure by simply adding some log output to its <code>postRestart</code> method. Make the <code>Register</code> type extend the <code>ActorLogging</code> trait and add the following method to it:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 </pre></td><td class="code"><pre><code class="scala">override def postRestart(reason: Throwable) { super.postRestart(reason) log.info(s"Restarted because of ${reason.getMessage}") } </code></pre></td></tr></table></div></figure>Now, if you send the two <code>Customer</code> actors a bunch of <code>CaffeineWithdrawalWarning</code> messages, you should see the one or the other of those log outputs, confirming that our <code>Register</code> actor has been restarted.<h3>Death of an actor</h3>Often, it doesn’t make sense to restart an actor again and again – think of an actor that talks to some other service over the network, and that service has been unreachable for a while. In such cases, it is a very good idea to tell Akka how often to restart an actor within a certain period of time. If that limit is exceeded, the actor is instead stopped and hence dies. Such a limit can be configured in the constructor of the supervisor strategy:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 </pre></td><td class="code"><pre><code class="scala">import scala.concurrent.duration._ import akka.actor.OneForOneStrategy import akka.actor.SupervisorStrategy.Restart OneForOneStrategy(10, 2.minutes) { case _ => Restart } </code></pre></td></tr></table></div></figure><h3>The self-healing system?</h3>So, is our system running smoothly, healing itself whenever this damn paper jam occurs? Let’s change our log output:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 </pre></td><td class="code"><pre><code class="scala">override def postRestart(reason: Throwable) { super.postRestart(reason) log.info(s"Restarted, and revenue is $revenue cents") } </code></pre></td></tr></table></div></figure>And while we are at it, let’s also add some more logging to our <code>Receive</code> partial function, making it look like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 </pre></td><td class="code"><pre><code class="scala">def receive = { case Transaction(article) => val price = prices(article) sender ! createReceipt(price) revenue += price log.info(s"Revenue incremented to $revenue cents") } </code></pre></td></tr></table></div></figure>Ouch! Something is clearly not as it should be. In the log, you will see the revenue increasing, but as soon as there is a paper jam and the <code>Register</code> actor restarts, it is reset to <code>0</code>. This is because restarting indeed means that the old instance is discarded and a new one created as per the <code>Props</code> we initially passed to <code>actorOf</code>.Of course, we could change our supervisor strategy, so that it resumes in case of a <code>PaperJamException</code>. We would have to add this to the <code>Barista</code> actor:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 </pre></td><td class="code"><pre><code class="scala">val decider: PartialFunction[Throwable, Directive] = { case _: PaperJamException => Resume } override def supervisorStrategy: SupervisorStrategy = OneForOneStrategy()(decider.orElse(SupervisorStrategy.defaultStrategy.decider)) </code></pre></td></tr></table></div></figure>Now, the actor is not restarted upon a <code>PaperJamException</code>, so its state is not reset.<h3>Error kernel</h3>So we just found a nice solution to preserve the state of our <code>Register</code> actor, right?Well, sometimes, simply resuming might be the best thing to do. But let’s assume that we really have to restart it, because otherwise the paper jam will not disappear. We can simulate this by maintaining a boolean flag that says if we are in a paper jam situation or not. Let’s change our <code>Register</code> like so:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 </pre></td><td class="code"><pre><code class="scala">class Register extends Actor with ActorLogging { import Register._ import Barista._ var revenue = 0 val prices = Map[Article, Int](Espresso -> 150, Cappuccino -> 250) var paperJam = false override def postRestart(reason: Throwable) { super.postRestart(reason) log.info(s"Restarted, and revenue is $revenue cents") } def receive = { case Transaction(article) => val price = prices(article) sender ! createReceipt(price) revenue += price log.info(s"Revenue incremented to $revenue cents") } def createReceipt(price: Int): Receipt = { import util.Random if (Random.nextBoolean()) paperJam = true if (paperJam) throw new PaperJamException("OMG, not again!") Receipt(price) } } </code></pre></td></tr></table></div></figure>Also remove the supervisor strategy we added to the <code>Barista</code> actor.Now, the paper jam remains forever, until we have restarted the actor. Alas, we cannot do that without also losing important state regarding our revenue.This is where the error kernel pattern comes in. Basically, it is just a simple guideline you should always try to follow, stating that if an actor carries important internal state, then it should delegate dangerous tasks to child actors, so as to prevent the state-carrying actor from crashing. Sometimes, it may make sense to spawn a new child actor for each such task, but that’s not a necessity.The essence of the pattern is to keep important state as far at the top of the actor hierarchy as possible, while pushing error-prone tasks as far to the bottom of the hierarchy as possible.Let’s apply this pattern to our <code>Register</code> actor. We will keep the revenue state in the <code>Register</code> actor, but move the error-prone behaviour of printing the receipt to a new child actor, which we appropriately enough call <code>ReceiptPrinter</code>. Here is the latter:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 </pre></td><td class="code"><pre><code class="scala">object ReceiptPrinter { case class PrintJob(amount: Int) class PaperJamException(msg: String) extends Exception(msg) } class ReceiptPrinter extends Actor with ActorLogging { var paperJam = false override def postRestart(reason: Throwable) { super.postRestart(reason) log.info(s"Restarted, paper jam == $paperJam") } def receive = { case PrintJob(amount) => sender ! createReceipt(amount) } def createReceipt(price: Int): Receipt = { if (Random.nextBoolean()) paperJam = true if (paperJam) throw new PaperJamException("OMG, not again!") Receipt(price) } } </code></pre></td></tr></table></div></figure>Again, we simulate the paper jam with a boolean flag and throw an exception each time someone asks us to print a receipt while in a paper jam. Other than the new message type, <code>PrintJob</code>, this is really just extracted from the <code>Register</code> type.This is a good thing, not only because it moves away this dangerous operation from the stateful <code>Register</code> actor, but it also makes our code simpler and consequently easier to reason about: The <code>ReceiptPrinter</code> actor is responsible for exactly one thing, and the <code>Register</code> actor has become simpler, too, now being only responsible for managing the revenue, delegating the remaining functionality to a child actor:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 </pre></td><td class="code"><pre><code class="scala">class Register extends Actor with ActorLogging { import akka.pattern.ask import akka.pattern.pipe import context.dispatcher implicit val timeout = Timeout(4.seconds) var revenue = 0 val prices = Map[Article, Int](Espresso -> 150, Cappuccino -> 250) val printer = context.actorOf(Props[ReceiptPrinter], "Printer") override def postRestart(reason: Throwable) { super.postRestart(reason) log.info(s"Restarted, and revenue is $revenue cents") } def receive = { case Transaction(article) => val price = prices(article) val requester = sender (printer ? PrintJob(price)).map((requester, _)).pipeTo(self) case (requester: ActorRef, receipt: Receipt) => revenue += receipt.amount log.info(s"revenue is $revenue cents") requester ! receipt } } </code></pre></td></tr></table></div></figure>We don’t spawn a new <code>ReceiptPrinter</code> for each <code>Transaction</code> message we get. Instead, we use the default supervisor strategy to have the printer actor restart upon failure.One part that merits explanation is the weird way we increment our revenue: First we ask the printer for a receipt. We map the future to a tuple containing the answer as well as the requester, which is the sender of the <code>Transaction</code> message and pipe this to ourselves. When processing that message, we finally increment the revenue and send the receipt to the requester.The reason for that indirection is that we want to make sure that we only increment our revenue if the receipt was successfully printed. Since it is vital to never ever modify the internal state of an actor inside of a future, we have to use this level of indirection. It helps us make sure that we only change the revenue within the confines of our actor, and not on some other thread.Assigning the <code>sender</code> to a <code>val</code> is necessary for similar reasons: When mapping a future, we are no longer in the context of our actor either – since <code>sender</code> is a method, it would now likely return the reference to some other actor that has sent us a message, not the one we intended.Now, our <code>Register</code> actor is safe from constantly being restarted, yay!Of course, the very idea of having the printing of the receipt and the management of the revenue in one place is questionable. Having them together came in handy for demonstrating the error kernel pattern. Yet, it would certainly be a lot better to seperate the receipt printing from the revenue management altogether, as these are two concerns that don’t really belong together.<h3>Timeouts</h3>Another thing that we may want to improve upon is the handling of timeouts. Currently, when an exception occurs in the <code>ReceiptPrinter</code>, this leads to an <code>AskTimeoutException</code>, which, since we are using the ask syntax, comes back to the <code>Barista</code> actor in an unsuccessfully completed <code>Future</code>.Since the <code>Barista</code> actor simply maps over that future (which is success-biased) and then pipes the transformed result to the customer, the customer will also receive a <code>Failure</code> containing an <code>AskTimeoutException</code>.The <code>Customer</code> didn’t ask for anything, though, so it is certainly not expecting such a message, and in fact, it currently doesn’t handle these messages. Let’s be friendly and send customers a <code>ComebackLater</code> message – this is a message they already understand, and it makes them try to get an espresso at a later point. This is clearly better, as the current solution means they will never know that they will not get their espresso.To achieve this, let’s recover from <code>AskTimeoutException</code> failures by mapping them to <code>ComebackLater</code> messages. The <code>Receive</code> partial function of our <code>Barista</code> actor thus now looks like this:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 </pre></td><td class="code"><pre><code class="scala">def receive = { case EspressoRequest => val receipt = register ? Transaction(Espresso) receipt.map((EspressoCup(Filled), _)).recover { case _: AskTimeoutException => ComebackLater } pipeTo(sender) case ClosingTime => context.system.shutdown() } </code></pre></td></tr></table></div></figure>Now, the <code>Customer</code> actors know they can try their luck later, and after trying often enough, they should finally get their eagerly anticipated espresso.<h3>Death Watch</h3>Another principle that is important in order to keep your system fault-tolerant is to keep a watch on important dependencies – dependencies as opposed to children.Sometimes, you have actors that depend on other actors without the latter being their children. This means that they can’t be their supervisors. Yet, it is important to keep a watch on their state and be notified if bad things happen.Think, for instance, of an actor that is responsible for database access. You will want actors that require this actor to be alive and healthy to know when that is no longer the case. Maybe you want to switch your system to a maintenance mode in such a situation. For other use cases, simply using some kind of backup actor as a replacement for the dead actor may be a viable solution.In any case, you will need to place a watch on an actor you depend on in order to get the sad news of its passing away. This is done by calling the <code>watch</code> method defined on <code>ActorContext</code>. To illustrate, let’s have our <code>Customer</code> actors watch the <code>Barista</code> – they are highly addicted to caffeine, so it’s fair to say they depend on the barista:<figure class="code"><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers">1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 </pre></td><td class="code"><pre><code class="scala">class Customer(coffeeSource: ActorRef) extends Actor with ActorLogging { import context.dispatcher context.watch(coffeeSource) def receive = { case CaffeineWithdrawalWarning => coffeeSource ! EspressoRequest case (EspressoCup(Filled), Receipt(amount)) => log.info(s"yay, caffeine for ${self}!") case ComebackLater => log.info("grumble, grumble") context.system.scheduler.scheduleOnce(300.millis) { coffeeSource ! EspressoRequest } case Terminated(barista) => log.info("Oh well, let's find another coffeehouse...") } } </code></pre></td></tr></table></div></figure>We start watching our <code>coffeeSource</code> in our constructor, and we added a new case for messages of type <code>Terminated</code> – this is the kind of message we will receive from Akka if an actor we watch dies.Now, if we send a <code>ClosingTime</code> to the message and the <code>Barista</code> tells its context to stop itself, the <code>Customer</code> actors will be notified. Give it a try, and you should see their output in the log.Instead of simply logging that we are not amused, this could just as well initiate some failover logic, for instance.<h2>Summary</h2>In this part of the series, which is the second one dealing with actors and Akka, you got to know some of the important components of an actor system, all while learning how to put the tools provided by Akka and the ideas behind it to use in order to make your system more fault-tolerant.While there is still a lot more to learn about the actor model and Akka, we shall leave it at that for now, as this would go beyond the scope of this series. In the next part, which shall bring this series to a conclusion, I will point you to a bunch of Scala resources you may want to peruse to continue your journey through Scala land, and if actors and Akka got you excited, there will be something in there for you, too.</div> Thu, 12 Dec 2013 14:00:12 -0000 The Neophyte's Guide to Scala Part 16: Where to Go from Here - Daniel Westheide http://danielwestheide.com/blog/2013/04/03/the-neophytes-guide-to-scala-part-16-where-to-go-from-here.html <div class="entry-content">As I have already hinted at in the previous article, the Neophyte’s Guide to Scala is coming to an end. Over the last five months, we have delved into numerous language and library features of Scala, hopefully deepening your understanding of those features and their underlying ideas and concepts.As such, I hope this guide has served you well as a supplement to whatever introductory resources you have been using to learn Scala, whether you attended the <a href="https://www.coursera.org/course/progfun">Scala course at Coursera</a> or learned from a book. I have tried to cover all the quirks I stumbled over back when I learned the language – things that were only mentioned briefly or not covered at all in the books and tutorials available to me – and I hope that especially those explanations were of value to you.As the series progressed, we ventured into more advanced territory, covering ideas like type classes and path-dependent types. While I could go on writing about more and more arcane features, I felt like this would go against the idea of this series, which is clearly targeted at aspiring neophytes.Hence, I will conclude this series with some suggestions of where to go from here if you want more. Rest assured that I will continue blogging about Scala, just not within the context of this series.How you want to continue your journey with Scala, of course, depends a lot on your individual preferences: Maybe you are now at a point where you would like to teach Scala to others, or maybe you are excited about Scala’s type system and would like to explore some of the language’s more arcane features by delving into type-level programming.More often than not, a good way to really get comfortable with a new language and its whole ecosystem of libraries is to use it for creating something useful, i.e. a real application that is more than just toying around. Personally, I have also gained a lot from contributing to open source projects early on.In the following, I will elaborate those four paths, which, of course, are not mutually exclusive, and provide you with numerous links to highly recommendable additional resources.<h2>Teaching Scala</h2>Having followed this series, you should be familiar enough with Scala to be able to teach the basics. Maybe you are in a Java or Ruby shop and want to get your coworkers excited about Scala and functional programming.Great, then why not organize a workshop? A nice way of introducing people to a new language is to not do a talk with lots of slides, but to teach by example, introducing a language in small steps by solving tiny problems together. Active participation is key!If that’s something you’d like to do, the Scala community has you covered. Have a look at <a href="http://www.scalakoans.org/">Scala Koans</a>, a collection of small lessons, each of which provides a problem to be solved by fixing an initially failing test. The project is inspired by the <a href="http://rubykoans.com/">Ruby Koans</a> project and is a really good resource for teaching the language to others in small collaborative coding sessions.Another amazing project ideally suited for workshops or other events is <a href="http://scalatron.github.com/">Scalatron</a>, a game in which bots fight against each other in a virtual arena. Why not teach the language by developing such a bot together in a workshop, that will then fight against the computer? Once the participants are familiar enough with the language, organize a tournament, where each participant will develop their own bot.<h2>Mastering arcane powers</h2>We have only seen a tiny bit of what the Scala type system allows you to do. If this small hint at the high wizardry that’s possible got you excited and you want to master the arcane powers of type-level programming, a good starting resource is the blog series <a href="http://apocalisp.wordpress.com/2010/06/08/type-level-programming-in-scala/">Type-Level Programming in Scala</a> by Mark Harrah.After that, I recommend to have a look at <a href="https://github.com/milessabin/shapeless">Shapeless</a>, a library in which Miles Sabin explores the limits of the Scala language in terms of generic and polytypic programming.<h2>Creating something useful</h2>Reading books, doing tutorials and toying around with a new language is all fine to get a certain understanding of it, but in order to become really comfortable with Scala and its paradigm and learn how to think the Scala way, I highly recommend that you start creating something useful with it – something that is clearly more than a toy application (this is true for learning any language, in my opinion).By tackling a real-world problem and trying to create a useful and usable application, you will also get a good overview of the ecosystem of libraries and get a feeling for which of those can be of service to you in specific situations.In order to find relevant libraries or get updates of ones you are interested in, you should subscribe to <a href="http://notes.implicit.ly/">implicit.ly</a> and regularly take a look at <a href="https://github.com/languages/Scala">Scala projects on GitHub</a>.<h3>It’s all about the web</h3>These days, most applications written in Scala will be some kind of server applications, often with a RESTful interface exposed via HTTP and a web frontend.If the actor model for concurrency is a good fit for your use case and you hence choose to use the <a href="http://akka.io">Akka</a> toolkit, an excellent choice for exposing a REST API via HTTP is <a href="http://spray.io/documentation/spray-routing/">Spray Routing</a>. This is a great tool if you don’t need a web frontend, or if you want to develop a single-page web application that will talk to your backend by means of a REST API.If you need something less minimalistic, of course, <a href="http://playframework.org">Play</a>, which is part of the Typesafe stack, is a good choice, especially if you seek something that is widely adopted and hence well supported.<h3>Living in a concurrent world</h3>If after our two parts on actors and Akka, you think that Akka is a good fit for your application, you will likely want to learn a lot more about it before getting serious with it.While the <a href="http://akka.io/docs/">Akka documentation</a> is pretty exhaustive and thus serves well as a reference, I think that the best choice for actually learning Akka is Derek Wyatt’s book <a href="http://www.artima.com/shop/akka_concurrency">Akka Concurrency</a>, a preliminary version of which is already available as a PDF.Once you have got serious with Akka, you should definitely subscribe to <a href="http://letitcrash.com/">Let It Crash</a>, which provides you with news and advanced tips and tricks and regarding all things Akka.If actors are not your thing and you prefer a concurrency model allowing you to leverage the composability of Futures, your library of choice is probably Twitter’s <a href="http://twitter.github.com/finagle">Finagle</a>. It allows you to modularize your application as a bunch of small remote services, with support for numerous popular protocols out of the box.<h2>Contributing</h2>Another really great way to learn a lot about Scala quickly is to start contributing to one or more open source projects – preferably to libraries you have been using while working on your own application.Of course, this is nothing that is specific to Scala, but still I think it deserves to be mentioned. If you have only just learned Scala and are not using it at your day job already, it’s nearly the only choice you have to learn from other, more experienced Scala developers.It forces you to read a lot of Scala code from other people, discovering how to do things differently, possibly more idiomatically, and you can have those experienced developers review your code in pull requests.I have found the Scala community at large to be very friendly and helpful, so don’t shy away from contributing, even if you think you’re too much of a rookie when it comes to Scala.While some projects might have their own way of doing things, it’s certainly a good idea to study the <a href="http://docs.scala-lang.org/style/">Scala Style Guide</a> to get familiar with common coding conventions.<h2>Connecting</h2>By contributing to open source projects, you have already started connecting with the Scala community. However, you may not have the time to do that, or you may prefer other ways of connecting to like-minded people.Try finding a local Scala user group or meetup. <a href="http://www.scala-tribes.org">Scala Tribes</a> provides an overview of Scala communities across the globe, and the <a href="http://lanyrd.com/topics/scala/">Scala topic at Lanyrd</a> keeps you up-to-date on any kind of Scala-related event, from conferences to meetups.If you don’t like connecting in meatspace, the <a href="https://groups.google.com/forum/#!forum/scala-user">scala-user mailing list/Google group</a> and the <a href="irc://irc.freenode.net/scala">Scala IRC channel on Freenode</a> may be good alternatives.<h2>Other resources</h2>Regardless of which of the paths outlined above you follow, there are a few resources I would like to recommend:<ul><li><a href="http://www.manning.com/bjarnason/">Functional Programming in Scala</a> by Paul Chiusano and Rúnar Bjarnason, which is currently available as an Early Access Edition and teaches you a lot more about functional programming and thinking about problems with a functional mindset.</li><li>The <a href="http://docs.scala-lang.org/">Scala Documentation Site</a>, which, for some reason, is not linked very prominently from the main Scala website, especially the available <a href="http://docs.scala-lang.org/overviews/">guides</a> and <a href="http://docs.scala-lang.org/tutorials/">tutorials</a>.</li><li><a href="http://nerd.kelseyinnis.com/blog/2013/01/07/resources-for-getting-started-with-functional-programming-and-scala/">Resources for Getting Started With Functional Programming and Scala</a> by Kelsey Innis contains a lot more helpful links about some of the topics we covered in this series.</li></ul><h2>Conclusion</h2>I hope you have enjoyed this series and that I could get you excited about Scala. While this series is coming to an end, I seriously hope that it’s just the beginning of your journey through Scala land. Let me know in the comments how your journey went so far and where you think it will go from here.</div> Mon, 16 Dec 2013 14:00:13 -0000