Cool toolz in Scalaz and Cats toolboxes

Scalar Conf 2016

Jan Pustelnik / @gosubpl

A word about myself

  • Working at GFT Poland as a Programmer, University of Lodz as a Lecturer
  • Education: Mathematician, Computer Programmer, DevOps / Admin
  • Mixed background: C++, C#, Ada, Pascal, Java, Scala, Go(lang), Python, Perl, Bash, Awk, Lisp, ML, Haskell, Visual Basic, R

A word about myself

  • Varied domain: Numerical Methods, Civil Engineering, Investment Banking, Retail Banking, Compiler Construction
  • Varied experience: Linux networking kernel modules, FreeBSD Jails, Networked embedded devices (x86 and Motorola 68000 assembly), Protocols design

http://xkcd.com/435/

Programming?

  • Grounding problem is the fundamental problem which we are solving in our work
  • Good abstractions are most vital for both
  • Abstractions always leak so we need to adapt them to context
  • Debugging is a very important but underrated skill

Why Scala

  • Love Scala because it allows to build bespoke abstractions that fit
  • Scala is more like a human language than a programming language
  • Growing a DSL up to meet your domain is so natural in Scala
  • Right abstractions allow to adapt to changing environment easily

Why Scala

  • The code is dense and meaningful
  • You can read and understand the entire codebase of a system
  • Scala blends seamlessly various paradigms - OO, FP, AOP
  • Scala has this "Quality without a name" (you know it when you see it)

Living with Scala

  • Dynamic ecosystem, tools change rapidly (e.g. recent work by @milessabin on an old issue SI-2712)
  • Libraries change (Slick 2 to Slick 3, Scalaz & Cats)
  • Right abstractions, so easy in Scala, help (e.g. withTransaction taking a code block, tuned to your needs)
  • Vibrant, helpful community (e.g. typelevel/cats gitter channel, thanks @mpilquist for your help)

Chisels, anyone?

    By Aerolin55 - Own work, CC BY-SA 3.0, https://commons.wikimedia.org/w/index.php?curid=8266918

Are you serious?

What is this?

By Arian Zwegers (Warsaw, Old Town Square) [CC BY 2.0 (http://creativecommons.org/licenses/by/2.0)], via Wikimedia Commons

What is this?

By Giramondo1 (Freudenstadt Town Square) [CC BY-SA 2.0 (http://creativecommons.org/licenses/by-sa/2.0)], via Wikimedia Commons

What is this?

By FruitMonkey Description=Llanharan Town Square, from Bridgend Road facing North. CC BY-SA 3.0

Hmm... a town/city square ...

Do you see a PATTERN there?

Design Patterns

  • Design Patterns are reusable abstractions
  • Coming from architecture (of builings, cities, etc.), not IT
  • Inspired by books on architecture by Christopher Alexander

Design Patterns

Some nice syntax extensions

We will approach Cats and Scalaz from the Patterns angle, starting with some nice syntax extensions

  • Show
  • Eq - === and =!= (cats) / =/= (scalaz)

There are more nice syntax helpers, e.g. Order, please check the documentation!

Scalaz imports

  • Scalaz is not modular, just do this 
    
import scalaz._
import Scalaz._
  • No need to import anything else
  • Can cause name clashes and general slowness (implicits!)

Show - Cats imports

  • Cats is a modular libary
  • need this import to add .show to standard types (Cats) 
    
import cats.syntax.show._

Show

  • showing a couple of things
  • same code for Cats and Scalaz, modulo the import 
    
import cats.syntax.show._

object Main extends App {

	println("abc".show)

	println(10.show)
}
          					
    
abc
10

Show

  • custom class will toString by default
  • the result is not very meaningful 
    
import cats.syntax.show._

class Foo(val s: String) {}

object Main extends App {
	val foo: Foo = new Foo("foo")

	println(foo)
}
          					
    
com.gft.scala.scalarconf.Foo@51d67073

Show

  • custom class will not Show by default
  • this results in compile time error 
    
import cats.syntax.show._

class Foo(val s: String) {}

object Main extends App {
	val foo: Foo = new Foo("foo")

	println(foo.show)
}
          					
    
ERROR: value show is not a member of
	com.gft.scala.scalarconf.Foo

Show - instance

  • we need to inject show into Foo using the Show typeclass
  • in other words - create an instance of this typeclass for a given type 
    
import cats.syntax.show._

class Foo(val s: String) {}

object Main extends App {
	val foo: Foo = new Foo("foo")

	implicit val fooShow: Show[Foo] =
 	new Show[Foo] {
      		def show(foo: Foo) = foo.s
    	}

	println(foo.show)
}
          					
    
foo
  • same code for Scalaz, minus the import

Show

  • if we have a toString defined in Foo, we can use it with Show 
    
import cats.syntax.show._

class Foo(val s: String) {
  override def toString = s
}

object Main extends App {
	val foo: Foo = new Foo("foo")

	implicit val fooFromToStringShow: Show[Foo] =
		Show.fromToString[Foo]

	println(foo.show)
}
          					
    
foo
  • this code is Cats specific

Show

  • but Scalaz version needs only a minor tweak
  • notice two minor differences 
    
class Foo(val s: String) {
  override def toString = s
}

object Main extends App {
  val foo: Foo = new Foo("foo")

  implicit val fooFromToStringShow: Show[Foo] =
    		Show.showFromToString[Foo]
  
  println(foo.show)
}
          					
    
foo

Interludium: what magic worked for Int and String?

  • Neither Int nor String required implementing an instance of a Show typeclass
  • That is because they are already defined
  • e.g. for Option most operations are in OptionInstances
  • Generally look for SthInstances, SthSyntax and SthOps
  • Mechanics in both libraries similar
  • Cats manual on Typeclasses

I have toString everywhere so why bother?

  • Moving some checks run-time -> compile time, less risk of unexpected fail when deployed
  • We are using type system more extensively to assure our programs are correct
  • We are reducing cognitive load on readers of the code - all usages are intentional
  • No surprises at run-time - if show is not defined, code will not compile
  • Scala has built in DI / AOP tooling in the shape of implicit / Typeclasses, that we can leverage here

This pattern seems weird...

  • Not exactly, this same trick is used by plain vanilla Scala...
  • ... in both Ordering[T] and Ordered[T] typeclasses
  • to convert from Java's Comparator/Comparable interfaces...
  • And those two typeclasses have implicit conversions between them... cool!
  • And this exact motivation is also present in Java's interfaces... but Java's syntax is more limited

Eq

  • === / =!= type aware check for equality / inequality
  • will cause compile error when types don't match (comparison is impossible)
  • for cats include the import below 
    
import cats.syntax.eq._

Eq

  • for Scalaz no import, use =/= instead of =!= and don't use /== as it has precedence issues 
    
import cats.syntax.eq._
object Main extends App {

  println("abc" === "cde")

  println("abc" =!= "cde")
//println("abc" =/= "cde")

  println(5 === 5)
//println(5 === "a") // don't
}
          					
    
false
true
true

=== for Foo in Cats

  • by implementing the right instance of Eq typeclass
  • The rest will by done by syntax / ops
  • Cats is really modular, Eq typeclass is in another library, called algebra which actually implements... algebra 
    
import cats.syntax.eq._
object Main extends App {

  implicit val fooEqInstance: Eq[Foo] =
    new Eq[Foo] {
      def eqv(lhs: Foo, rhs: Foo): Boolean = lhs.s === rhs.s
    }

  println(foo === foo)
}          					
    
true

=== for Foo in Scalaz

  • by implementing the right instance of Equal typeclass
  • The rest, as usual, will by done by syntax / ops 
    
object Main extends App {
  implicit val fooEqualInstance: Equal[Foo] =
    new Equal[Foo] {
      def equal(lhs: Foo, rhs: Foo): Boolean =
		lhs.s === rhs.s
    }

  println(foo === foo)
}          					
    
true

Context and Functor

  • Imagine you would like to separate functions that operate on certain types (like f : A => B) from the computational context (environment) in which those functions are defined
  • Example could be a transformation of numbers into other numbers that you would like to unit test using scalacheck
  • But run in distributed environment

Context and Functor

  • So for unit testing your function might be running in a context of a simple list (transforming lists of numbers into lists of numbers)
  • but for real execution you would use Stream or Future[List] or List[Future] some other fancy Reactive thingy or modern Big-Data stuff - Spark RDD
  • Plus you get for free the re-usability of the function code, so much coveted in the OO world

Context and Functor

  • The OO GOF book fans should recognise here the Command pattern
  • enhanced with the Function Object pattern approach
  • and a bit of an Iterator pattern
  • we call this pattern simply Functor

The map function

Interface satisfying requirements stated above one of the map function


def map[A, B](f: A => B): F[B]

Implementation is every generic collection F that can be parametrised with types A and B as F[A] and F[B] supplied with map function that operates on F[A], takes functions going from A to B and returns F[B], applying f to each element of F[A] and building F[B] from results of this application

The map function

  • In Scala standard library there is a suitable map construct defined for TraversableLike trait, which sits near the root of inheritance tree of Scala Collections library 
    
 def map[B, That](f: A => B)
	(implicit bf: CanBuildFrom[Repr, B, That]): That = {
    def builder = { 
      val b = bf(repr)
      b.sizeHint(this)
      b
    }
    val b = builder
    for (x <- this) b += f(x)
    b.result
  }

The map function

But to actually use this for our custom type, we would have to overcome a couple of obstacles

  • inherit from TraversableLike
  • implement suitable CanBuildFrom for builder
  • think about what is necessary to implement all remaining methods of TraversableLike

Factoring out / abstraction

Can we?

  • treat this Context / map thing as a pattern
  • factor out only the part connected with the map function
  • define in abstract way some laws that this construct should obey to satisfy the "map applies f to each element of F[A] and builds F[B]" without "emulating" the definition using builder, as is the case with TraversableLike.map?

YES!

Functor

  • Cats and Scalaz introduce the Functor pattern
  • (Almost) everything that has map can be converted to Functor
  • Relevant implicit convertion needs to be introduced 
    
implicit val optionFunctor: Functor[Option] = 
new Functor[Option] {
  def map[A,B](fa: Option[A])(f: A => B) = fa map f
}

implicit val listFunctor: Functor[List] =
new Functor[List] {
  def map[A,B](fa: List[A])(f: A => B) = fa map f
}

Functor

  • In Cats the conversions itself are in OptionOps and ListOps
  • In Scalaz - OptionSyntax and ListSyntax (Ops & Syntax seem to have inverted meanings)
  • The method map itself is injected in both Scalaz & Cats in OptionInstances and ListInstances
  • You need the conversion to be in the scope, will not happen by itself

Functor laws

  • Functors are very common
  • But not all collections are functors
  • Functor needs to obey FunctorLaws (e.g. cats.laws.FunctorLaws) 
    
  def covariantIdentity[A](fa: F[A]): IsEq[F[A]] =
    fa.map(identity) <-> fa

  def covariantComposition[A, B, C]
      (fa: F[A], f: A => B, g: B => C): IsEq[F[C]] =
    fa.map(f).map(g) <-> fa.map(f andThen g)
  • e.g. Set is not a true functor (scalaz.outlaws.std.set._)
  • We call them lawless or outlaws
  • Nonetheless you can use implicit just for syntax benefit only if careful

Using Functors

  • You can now separate function from the context
  • And test your function separately from usage 
    
def transformWithFunc[F[_], A, B](a: F[A], fa: A => B)
	(implicit F: Functor[F]) = F.map(a)(fa)

def twice(x: Int) = 2 * x

println(transformWithFunc(List(1, 2, 3), twice))

List(2, 4, 6)

println(transformWithFunc(1.some, twice))

Some(2)

Composing Functors

  • Neat trick about Functors is that you can compose them with |@| the Scream operator
  • This is Cats code 
    
import cats.syntax.cartesian._

val oa: Option[Int] = Some(3)
val ob: Option[Int] = Some(5)

val od = oa |@| ob
val odx = od.map(_ + _)
println(odx)

						
    
Some(8)

Composing Functors

  • This operator is called a Cartesian in Cats... 
    
import cats.syntax.cartesian._

val oe = 3.some |@| 5.some
val oex = od.map(_ + _)
println(oex)

						
    
Some(8)

Composing Functors

  • ... and Apply in Scalaz
  • |@| is called here ApplicativeBuilder
  • note that instead of map we use apply to apply the Apply :) 
    
val oe = 3.some |@| 5.some
val oex = od.apply(_ + _)
println(oex)

						
    
Some(8)

Yes, this is a type!

Cats


def buildTwo[F[_], B](a: F[B], b: F[B])
	(implicit F: Cartesian[F]) = a |@| b

Scalaz


def buildTwo[F[_], B](a: F[B], b: F[B])
	(implicit F: Apply[F]) = a |@| b

Applicative Functor

  • Applicative Functor is a Functor that has a constructor
  • This constructor is called pure (or sometimes point) 
    
def pure[A](x: A): F[A]
						
    

Applicative[Option].pure(1)
						
    

Some(1)
						
    

Applicative[List].pure(1)
						
    

List(1)

Using Applicatives

  • Notice the catz.* import
  • Cats 0.4.1 does not contain applicatives yet, had to copy by hand from git repository
  • This code works for Scalaz when you replace the import with general Scalaz imports
  • And map with apply (name Applicative is the same for both libs 
    
import catz.syntax.applicative._

def sequenceA[F[_]: Applicative, A](list: List[F[A]])
	: F[List[A]] = list match {
    case Nil => (Nil: List[A]).pure[F]
    case x :: xs => (x |@| sequenceA(xs)).map { _ :: _ }
}

Using Applicatives

  • Notice what happens when None is encountered
  • Also, commented out line will not compile 
    
println(sequenceA(List(1.some, 2.some, 3.some)))

println(sequenceA(List(1.some, none[Int], 2.some)))

println(sequenceA(List(1.some, "a".some)))

//println(sequenceA(List(1.some, List(2))))

						
    
Some(List(1, 2, 3))
None
Some(List(1, a))

Why None?

  • Think about the addition example 
    
val on = 1.some |@| none[Int]

println(on.map(_ + _))
  • You cannot add an empty value... 
    
None
  • This makes sense...

Applicatives Pattern - Configuration

  • Reading a config file
  • Suppose you read a simple props file getting a list of lines
  • One configuration is correct, the other is not (it happens) 
    
val propsStringGood =
	List("db.user=gosubpl", "db.pass=pass", "db.name=dbase1")
val propsStringBad = List("db.user=gosubpl", "db.pass=pass")

println(propsStringGood.find(_.startsWith("db.user="))
	.map(_.replace("db.user=", "")))
println(propsStringBad.find(_.startsWith("db.name="))
	.map(_.replace("db.name=", "")))
						
    
Some(gosubpl)
None

Applicatives Pattern - Configuration

  • Here comes first nice usage of applicatives (works both for Scalaz and Cats) 
    
def readProperty(prop: String)(props: List[String])
	: Option[String] =
	props.find(_.startsWith(prop + "="))
	.map(_.replace(prop + "=", ""))

val dbUser = readProperty("db.user") _
val dbName = readProperty("db.name") _
val dbPass = readProperty("db.pass") _

def dbProps(config: List[String]) =
	(dbUser(config) |@| dbName(config) |@| dbPass(config))

Applicatives Pattern - Configuration

  • Now let's extract the configs (remember about apply for Scalaz) 
    
// use apply instead of map for Scalaz
println(dbProps(propsStringGood)
	.map(_ + " " + _ + " " + _))
println(dbProps(propsStringBad)
	.map(_ + " " + _ + " " + _))						
						
    
Some(gosubpl dbase1 pass)
None

Applicatives Pattern - Configuration

  • Or in a simpler way with sequenceA (remember that one?) 
    
def applyProps(f : Function1[List[String], Option[String]],
	s: List[String])
	 = f(s)

println(sequenceA(List(dbUser, dbName, dbPass)
	.map(applyProps(_, propsStringGood))))
						
    
Some(List(gosubpl, dbase1, pass))

Applicatives Pattern - Traverse

  • Fortunately, it is in the library by default
  • In the form of sequence and (more generally) traverse 
    
println(List(dbUser, dbName, dbPass)
	.map(applyProps(_, propsStringGood)).sequence)

println(List(dbUser, dbName, dbPass)
	.map(applyProps(_, propsStringGood))
	.traverse(identity))

println(List(dbUser, dbName, dbPass)
	.map(applyProps(_, propsStringGood))
	.traverse(_.map(_.toUpperCase())))

						
    
Some(List(gosubpl, dbase1, pass))
Some(List(gosubpl, dbase1, pass))
Some(List(GOSUBPL, DBASE1, PASS))

Validations

  • And if I wanted to know which part of the config failed? 
    
val propsStringBad = List("db.user=gosubpl", "db.pass=pass")
val propsStringVeryBad = List("db.user=gosubpl")
	
						
    
None
None

Validated / Validation

  • You can use the Validated (Cats) or Validation (Scalaz) ADTs
  • Not exactly Scala syntax
  • Please refer to @rabbitonweb 's presentation
  • And #scalarconf 's blog entry on ADTs by GFTs @marcinkowalski
  • Cats 
    
import cats.data.Validated

Validated[SuccessType,FailureType] = Valid | Invalid
  • Scalaz 
    
Validation[SuccessType,FailureType] = Succes | Failure

Validation

  • let's start with Scalaz for a change (good old trusty Scalaz...) 
    
def readPropertyV(prop: String)(props: List[String])
	: Validation[String, String] =
  props.find(_.startsWith(prop + "="))
	.map(_.replace(prop + "=", ""))
  match {
     case None =>
     	("Property " + prop + " not found in config")
	.failure[String]
     case Some(s) => s.success[String]
  }

val dbUserV = readPropertyV("db.user") _
val dbNameV = readPropertyV("db.name") _
val dbPassV = readPropertyV("db.pass") _

Validation

  • Let's run this (this code will serve both for Scalaz and Cats) 
    
val propsStringBad =
	List("db.user=gosubpl", "db.pass=pass")
val propsStringVeryBad = List("db.user=gosubpl")

def applyPropsV(f: Function1[List[String],
	Validation[String, String]],
	s: List[String])
	= f(s)

println(List(dbUserV, dbNameV, dbPassV)
	.map(applyPropsV(_, propsStringBad))
	.traverseU(identity))

println(List(dbUserV, dbNameV, dbPassV)
	.map(applyPropsV(_, propsStringVeryBad))
	.sequenceU)
						
    
Failure(Property db.name not found in config)
Failure(Property db.name not found in config
	Property db.pass not found in config)

Validated

  • And now Cats
  • There is a convenience method fromOption :) 
    
def readPropertyV(prop: String)(props: List[String])
	: Validated[String, String] =
    Validated.fromOption(
    	props.find(_.startsWith(prop + "="))
	.map(_.replace(prop + "=", "")),
		"Property " + prop
		+ " not found in config")

val dbUserV = readPropertyV("db.user") _
val dbNameV = readPropertyV("db.name") _
val dbPassV = readPropertyV("db.pass") _

def applyPropsV(f: Function1[List[String],
	Validated[String, String]],
	s: List[String])
	= f(s)

Where to go from here?

Conference talks!

Where to go from here?

Useful links

Thank you




Jan Pustelnik

Slides: http://gosubpl.github.io/scalar-2016-slides