Setting MySQL Load Balancing in Play and Scala

April 11th, 2012 — 1:34pm

In my previous post I showed a way to set MySQL load balancing with Java and Spring.

To make your queries go to the slave database, you need to set connection to read-only. In Play Framework 2.0 with Scala, this can’t be simpler:

def withReadOnlyConnection[A](block: Connection => A)(implicit app: Application): A = {
    DB.withConnection {
      connection =>
        try {
        } finally {

Now you can just wrap your database code as:

withReadOnlyConnection[Option[User]] {
   implicit connection: Connection =>

      etc, etc.

For additional info on Play and Anorm see: Anorm, simple SQL data access

Comment » | Software

String class broken in latest JDK

September 30th, 2010 — 11:32am

Comment » | Software

Merging EOL and Freebase

August 25th, 2010 — 5:41pm

I wrote a simple Acre application that allows users to easily add EOL keys to Freebase organism classification.

Encyclopedia of Life is an ambitious project to create “.. electronic page for each species of organism on Earth..” and Freebase is a open structured data repository recently purchased by Google.

Go to app: (you need to have Freebase account)

Comment » | Software

ActiveMQ flow control and Apache Camel transacted route gotchas

July 9th, 2010 — 9:02pm

We have a system that uses Apache Camel and ActiveMQ. It handles periodic bursts of 20,000 messages. The end consumer is slow and it takes about an hour to process all the messages. On their route, messages are passed from one queue to another. On our production system we stumbled onto an unexpected problem. After finishing few thousand messages the whole system would freeze.

The problem was with ActiveMQ flow control and transacted routes in Camel.

When you have slow consumers, to prevent queues from growing infinitely, ActiveMQ has a limit of how many messages you can put in a queue. When limit is reached, producer is, by default, forced to wait until the resources free. You can set these limits on system level or per queue level. Problem arises when you have transacted Camel route and system limit is reached.

AMQ Flow Control

Route that moves messages from queue A to queue B is inside a JMS transaction – you can’t remove message from queue A until the message is successfully placed on queue B.

If system limit is reached, no new messages can be sent to any queue. So, producers are forced to wait, transaction doesn’t complete, messages can’t be taken off queue A and no resource gets freed. The  whole processing freezes.

There are numerous ways you can work around this problem. You can turn off flow control and potentially let queues grow indefinitely.

In our case, solution was to set per-queue limits so that system limits can never be reached. Sum of limits for all queues needs to be less than the system limit. That way, as consumer takes messages from queue B, new messages can come in, transactions can complete and messages can be taken off queue A. Messages are consumed from queues A and B at the same pace and the whole system works fine.

In our case, I’ve set memoryLimit to 10m for our 13 queues and system memoryUsage to 180m.

For details see :

2 comments » | Software

Solr DisMax parser and stop words

May 25th, 2010 — 11:49pm

If you want to use DisMax parser in Solr you need to be careful how to index the fields that DisMax will be using.

If you mix fields that filter out stop words (plain text) and fields that do not filter out stop words (like author names), your simple queries might end up with no results.

By default, DisMax will display only results that contain all the words from your query string. If your query has stop words like “ants of madagascar”, stop word “of” might not be found in any of the fields – it’s not in author names and it’s filtered out in article body – and you will get zero results.

Possible workarounds:

  • Relax Minimum Match (mm) requirement.
    Downside: Lowering mm will increase number of results. mm of 50% on “ants madagascar” will return all documents that have “ants” and all docs that have “madagascar” in them.
  • Do not filter out stop words.
    Downside: Your index can get large and you might get large number of less relevant results.
  • Use other indexing schemes like N-Grams.

This article explains the details.

Also see this and this discussion.

Comment » | Software

XSLT Unicode Horror

May 18th, 2010 — 10:50pm

3 comments » | Software

Nasty bug with generics and introspection

April 19th, 2010 — 5:15pm

Comment » | Software

RDFa on PLoS

October 24th, 2009 — 8:10am

8 comments » | Software

Hoy Kid

October 17th, 2009 — 6:43pm

Laurie wrote another article for Mission Local, San Francisco Mission neighborhood blog/news site. Hoy Kids is turning into a regular column. And my baby boy is on the front page !

Read the article>

Comment » | Mission, Web

Birth – A Play by Karen Brody

September 5th, 2008 — 10:40pm

Laurie performed last weekend in play Birth by Karen Brody and directed by Aimée Miles. It was a performance for the Birth Fest 2008 in San Francisco. Birth Fest is an event created to support midwifery and natural birth.

Laurie got an additional applause after showing her belly !

2 comments » | Art, Society

Back to top