Mindfood

Most programmers know several languages. We, as web developers, work on a daily basis with Groovy, SQL, Bash Scripting, HTML, CSS, JavaScript…

Learning different languages allows us to solve problems with the most appropriate language and to explore new paths of thinking about computational problems.

Java has long been known simply as a programming language. Today, when thinking in Java we refer also a robust and mature development platform. Currently, the Java Virtual Machine (JVM) supports over 200 different programming languages.

These days we have been exploring different languages on the JVM: Java, Groovy, Scala, Clojure, JRuby… and how to incorporate them into a Grails project. We shared our experiences with others developers at Spring I/O 2012 and Codemotion 2012.

Type conversion the standard way

Type conversion or casting is a programming language method for changing an object’s data type into another.

I’m sure you are familiar with this code that converts a String to an Integer.

def number = (Integer)'1'
def number = '1'.toInteger()
def number = '1' as Integer

If I want to convert the type of my own objects, I need to create a method to achieve this goal. I copy object properties to another object in a generic way; if a property exists on target object, I copy it from the source object.

class User {
    String name
    String city
    Integer age

    def toAdminUser() {
        def adminUser = new AdminUser()
        copyProperties(this, adminUser)
        return adminUser
    }

    def copyProperties(source, target) {
        source.properties.each { key, value ->
            if (target.hasProperty(key) && !(key in ['class', 'metaClass'])) {
                target[key] = value
            }
        }
    }
}

class AdminUser {
    String name
    String city
    Integer age
}

Now I can do something like this:

adminUser = user.toAdminUser() 

Type conversion the fancy way

Great, but I want to use this fancy way to coerce one type to another:

adminUser = user as AdminUser 

Simple, Groovy supports operator overloading and creating your own type conversion is really easy: we can override the asType() method.

class User {
    String name
    String city
    Integer age

    Object asType(Class clazz) {
        if (clazz == AdminUser) {
            def adminUser = new AdminUser()
            copyProperties(this, adminUser)
            return adminUser
        }
        else {
            super.asType(clazz)
        }
    }

    def copyProperties(source, target) {
        source.properties.each { key, value ->
            if (target.hasProperty(key) && !(key in ['class', 'metaClass'])) {
                target[key] = value
            }
        }
    }
}

If you wrote multithreaded code in Groovy/Grails, I’m sure you stumbled upon the GPars library. It simplifies parallel processing - sometimes it’s as easy as wrapping a code fragment with a GPars closure, changing the iteration method (in our example from each to eachParallel) and passing the closure to a GPars pool. The library implementation handles the executor pool creation and adds new methods to collections.

Whenever I deliberately add multithreading, I always separate responsibilities (processing logic and concurrency handling). I wrap the single-threaded class with a new one, handing the parallel execution (creating a pool, managing threads, handling cancellation, etc.):

class MultithreadedProcessorService {
    ProcessorService processorService
    def threadPoolSize = ConfigurationHolder.config.threadPool.size

    void processMultithreaded(batches) {
        GParsPool.withPool(threadPoolSize) {
            batches.eachParallel { batch ->
                processorService.process(batch)
            }
        }
    }
}

class ProcessorService {
    void process(batch) {
        // some processing
    }
}

To test the multithreaded class you can you use Spock interactions. As opposed to Groovy and Grails mocks, they are thread-safe and you won’t get any weird and unstable results:

class MultithreadedProcessorServiceSpec extends UnitSpec {
    private MultithreadedProcessorService multithreadedProcessorService
    // Using Spock mocks because they are thread-safe
    private ProcessorService processorService = Mock(ProcessorService)

    def setup() {
        mockConfig('threadPool.size=2')
        multithreadedProcessorService = new MultithreadedProcessorService(
            processorService: processorService
        )
    }

    def 'processes batches using multiple threads'() {
        given:
        def batches = [[0, 1], [2, 3, 4], [5]]
        def processedBatches = markedAsNotProcessed(batches)

        when:
        multithreadedProcessorService.processMultithreaded(batches)

        then:
        batches.size() * processorService.process({ batch ->
            processedBatches[batch] = true
            batch in batches
        })
        assertAllProcessed(processedBatches)

    }

    private markedAsNotProcessed(batches) {
        batches.inject([:]) { processed, batch ->
            processed[batch] = false
            processed
        }
    }

    private void assertAllProcessed(batches) {
        assert batches*.value == [true] * batches.size()
    }
}

This post series present the best of Osoco tests - tests that were tricky or we are just proud of. You can find a runnable source code for this test and more in the Grails Test Gallery project shared on GitHub.

In our projects we include at least one integration test per domain class. We want to assure us that the GORM mapping is correct and the class is persistable in the target database. The test plan is simple:

• create a valid persistable sample of the domain object
• save it
• clear Hibernate first level cache (i.e. flush and clear current session)
• retrieve the object once again and compare it with the previously saved object. They must be equal as by equals() but be different object references

We put the steps above in a base abstract specification. In concrete specifications we implement merely the factory method spawning a persistable object instance.

abstract class PersistenceSpec extends IntegrationSpec {
    SessionFactory sessionFactory

    protected abstract createPersistableDomainObject()

    def 'persistable domain object should be able to be saved and retrieved'() {
        given:
        def persistableDomainObject = createPersistableDomainObject()

        when:
        def savedDomainObject = persistableDomainObject.save()

        then:
        savedDomainObject.id != null

        when:
        clearFirstLevelCache()

        def retrievedDomainObject = persistableDomainObject.class.get(savedDomainObject.id)

        then:
        savedDomainObject == retrievedDomainObject
        !savedDomainObject.is(retrievedDomainObject)
    }

    private clearFirstLevelCache() {
        sessionFactory.currentSession.flush()
        sessionFactory.currentSession.clear()
    }
}

This post series present the best of Osoco tests - tests that were tricky or we are just proud of. You can find a runnable source code for this test and more in the Grails Test Gallery project shared on GitHub.

Hace unos meses, mi compañero Diego Toharia escribió sobre el uso de la herramienta Screen en máquinas remotas. Vamos a repasar algunos conceptos básicos y aprender algunas opciones más avanzadas para poder automatizar nuestro trabajo.

Uso básico

Crear una nueva sesión de screen:

$ screen -S sessionname

Reanudar una sesión detached:

$ screen -r sessionname

Listar todas las sesiones de screen:

$ screen -ls

Eliminar una sesión:

$ screen -S sessionname -X quit

Eliminar todas las sesiones de screen:

$ killall screen

Atajos de teclado útiles:

Ctrl + a, d Detach screen del terminal
Ctrl + a, c Crear una nueva ventana
Ctrl + a, space Cambiar a la ventana siguiente
Ctrl + a, backspace Cambiar a la ventana anterior
Ctrl + a, A Renombrar la ventana actual
Ctrl + a, " Listar todas ventanas para seleccionar una
Ctrl + a, k Eliminar la ventana actual
Ctrl + a, ? Mostrar los atajos de teclado

Múltiples shells abiertas en la misma terminal:

Ctrl + a, | Dividir verticalmente
Ctrl + a, S Dividir horizontalmente
Ctrl + a, tab Cambiar el foco a la siguiente región
Ctrl + a, X Eliminar la región actual
Ctrl + a, Q Eliminar todas la regiones menos la actual

Una sesión tranquila

$ ssh me@server.com
$ screen -S sessionname
$ start something really important

Desconexión del servidor (pánico en otros tiempos sin utilizar screen).

$ ssh me@server.com
$ screen -r sessionname

Todo correcto, continuamos trabajando.

Automatizar el trabajo

Después de una desconexión, podemos volver a autenticarnos en nuestra maquina remota y recuperar la sesión en un simple paso:

$ ssh me@server.com -t "screen -r sessionname"

¿Cómo podemos mejorar esto? autossh es la respuesta.

autossh, automáticamente reconecta una sesión SSH y recupera una sesión de forma transparente:

$ ssh me@server.com
$ screen -S sessionname
$ Ctrl + a, d
$ exit
$ autossh me@server.com -t "screen -r sessionname"

De hecho, autossh incluye un script llamado rscreen para sesiones perpetuas.

OK, funciona. Pero no parece especialmente divertido porque primero necesitamos conectarnos al servidor, crear una nueva sesión de screen, hacer detached de la misma, desconectarnos del servidor y finalmente conectarnos de nuevo con autossh.

¿Qué podemos hacer? Si usamos screen -R recuperamos la sesión o incluso la creamos primero si no existe, finalmente hemos solucionado todos los problemas.

$ autossh me@server.com -t "screen -R sessionname"

If you work with GORM and your objects are held in collections, you should implement equals and hashCode methods. As everything, they should be thouroughly tested if they obey equals-hashCode contract. At Osoco we created a equals-hashcode-test Grails plugin that provides a base Spock specification for that. In your equals and hashCode tests you simply have to extend the EqualsHashCodeSpec and provide at least two methods:

• a factory method createDomainObjectToCompare that will create a fresh object for the comparison
• a map of modified properties included in equals and hashCode
• optionally a map of modified properties ignored in both methods

Additionally in case of inheritance, we check whether the equals method complies with the symmetry rule. We do it in an extra feature method added to the spec:

def 'equals relation must be symmetric with inheritance'() {
    given:
    def content = new Content()

    and:
    def book = new Book()

    expect:
    (content == book) == (book == content)
}

You may say, in Grails 2.0 and Groovy 1.8 there are @Equals and @HashCode AST transformations and you don’t need to write tests of them. I agree to some point. I think you can trust Groovy developers and you don’t have to test the methods for reflexivity, symmetry, transivity, consistency, and ‘non-nullity’. Anyway, you have to test the annotation configuration if it includes the properties that actually account for the logical equality between objects.

In the future releases of the plugin we will surely simplify the base specification and remove some checks but we will surely continue testing both methods (with our plugin it’s dirt-cheap!)

This post series present the best of Osoco tests - tests that were tricky or we are just proud of. You can find a runnable source code for this test and more in the Grails Test Gallery project shared on GitHub.

My opinion about magic in software development coincides with these great posts:

Do You Believe In Magic?

Magic in software development

I extract some fragments of them that I would like to highlight.

Bad word: Magic

I often find that people talk about magic when referring to programming languages and frameworks.

The concept that some particular language features are more magical than others is insane. None of it involves fairy dust or anything particularly complicated; it’s all just programming.

There is absolutely no logic in calling language features magical. It implies that certain programming techniques are inherently fantastic and strange. That’s only the case if you don’t understand the language.

The correct response is not calling arbitrary features magic. The correct response is to keep learning how to write code and overcome your ignorance.

Magic rocks!!

Ultimately, you can continue using magic, but what’s magic? I’d define it as anything below your current level of abstraction that you don’t understand.

Any sufficiently advanced technology is indistinguishable from magic.

Arthur C. Clarke

Magic is a good idea, we don’t understand exactly how quantum physics works but we accept that it does and make good use of it.

Now, if you don’t understand some feature of your favorite framework or how it works behind the scenes, don’t worry. Do not be intimidated, just learn and enjoy without worrying about every detail.

The diversity of languages and programming paradigms allow us to solve existing problems by thinking solutions from very different approaches.

But why a Groovy developer should learn functional programming?

As I learned about functional programming, I found good ideas but I also found that it brought new clarity to my thinking about the design of class and methods. It also allowed me to write more concise code and how to make it better for reuse.

A few weeks ago, I spoke about functional programming with Groovy at Greach 2011. I presented basic concepts of functional programming paradigm and then focus on how to apply them to improve the code that we write with Groovy.

In our application we have to consume JMS messages. We install Grails JMS plugin and write a listener for this task. To not violate SRP, our consumer receives a message and routes it to a MessageProcessorService for further processing.

Curious? Continue reading on Marcin’s weblog…

Sometimes you need to run Grails embedded Tomcat container in background as a part of your build in Jenkins/Hudson. Launching a grails run-app & as Execute shell build step will result in a warning from Jenkins: Process leaked file descriptors.

The problem is described in the Jenkins wiki with some possible solutions. I chose the daemonize tool. As far as I know it is not available in any Debian repository, so I had to build it myself. It takes less than 3 minutes; just follow the instructions from the daemonize website.

I wrapped the grails run-app with daemonize in a handy script: run-app-as-daemon.sh that you can download from the Osoco’s GitHub repository.

The script starts the embedded Grails server with HTTP or HTTPS connector and then checks if the server is up and running by polling a configured URL. After e.g. some functional tests, it kills the server process by its PID.

If you want to apply the script in your build, you have to customize some variables, listed (and documented) in the script header. Usage is very simple:

run-app-as-daemon.sh start|start-https|stop|force-stop

Sample Jenkins build configuration:

Build steps:

your-app-dir/run-app-as-daemon.sh force-stop
your-app-dir/run-app-as-daemon.sh start

Post build tasks:

your-app-dir/run-app-as-daemon.sh stop