Logging issue

The situation

Everything was working fine, well almost. I figured out that sometime in my dispatcher, the logger (getLogger()) wasn’t working and i needed to use logging directly. I didn’t ask too much question at the time. Today, i’m not sure what i changed, but the logging wasn’t working anymore.

The problem

Usually, i put at the top of the file the logger

logger = logging.getLogger('PROJECT.' + __name__)

It took me a bit of time to realize that the setLevel() from the settings (tornado) was done AFTER the init of the file, so everything in my dispatcher was set at the default level (info)

The solution

Maybe it’s not the best way to do it, but just put it just before using it. Everything is working fine now

class Dispatcher(tornado.web.RequestHandler):
    def initialize(self):
        logger = logging.getLogger('PROJECT.' + __name__)
        logger.debug("initialize")
    ...

Tornado coroutine won’t catch my exception

The problem

I want to catch my exception only at one place in my dispatcher (routing) and not in every route. So the trick was to put the call to the handler in a try-except. But it doesn’t work …. why ?

try:
    yield func(*args **kwargs)
except AccessException, e:
    logger.exception(e.message)
    raise tornado.web.HTTPError(403)
except Exception, e:
    logger.exception(e.message)
    raise tornado.web.HTTPError(500)

So if func() raise a AccessException, it doesn’t work but if i raise it directly in the try block, it’s working.

The solution

Yes, beginner’s mistake … Tornado coroutine doesn’t catch BaseException, only Exception

class AccessException(Exception):
    pass

i’m using python 2.7 and tornado 4.3

Mongodb: Update nested array using positional operator

This is more or less my current schema:

{ "_id" : 1
  "user_id": 1,
  "message" : "Yes"
  "translations" : [
    { 
      "destination" : "fr",
      "text": "Oui",
      "rating" : [
        { "user_id" : 1,
          "rating" : 1
        },
        { "user_id" : 2,
          "rating" : 1
        }
      ]
    }
  ]
}

The problem:

At the moment (3.2) there’s no way to update a nested array using the positional operator. This is not new. It’s been more then 4-5 years … So how can i update a rating for a specific user & destination ?

The positional $ operator cannot be used for queries which traverse more than one array, such as queries that traverse arrays nested within other arrays, because the replacement for the $ placeholder is a single

There’s an issue on Jira related to this for a long time and there’s not solution for now and it’s in the backlog (not assigned to anyone). The desired version is 3.3 so we may have a solution, but it was supposed to be on 1.9 or 2.1 version and after 2.4 or 2.6, so don’t expect anything until it’s done. You can also find an exemple of the problems here on gist.

db.getCollection('message').update({
    '_id': 1,
    'translations.destination': 'fr',
    'translations.rating.user_id':'1'
},
{
    '$set': {
        "translations.$.rating.$.rating": 5
    }
}

Too many positional (i.e. ‘$’) elements found in path ‘translation.$.rating.$.rating’

The possible solutions:

As you expect, there’s no easy solution for now. That’s why it’s important to design your DB properly at the beginning.

You can always use the position in the array if you know which one you want to modify. So in this case, this will update the rating of this user_id on the first translation. It means, 2 queries, the first to know which translation you want to update and after update the rating using this.

“translations.0.rating.$.rating”: 5

In my case since there’s a limited number of translations possible (language), i decided to restructure my collection and using translations as a dict/obj.

{ "_id" : 1
  "user_id": 1,
  "message" : "Yes"
  "translations" : {
    "fr": { 
      "destination" : "fr",
      "text": "Oui",
      "rating" : [
        { "user_id" : 1,
          "rating" : 1
        },
        { "user_id" : 2,
          "rating" : 1
        }
      }
    },
    "en": {...}
  ]
}

So you can do this directly:

“translations.fr.rating.$.rating”: 5

You can find more information on SO here , here and here

MongoDB: sorting issue (case sensitive)

The problem

Sadly, at the moment ( 3.2 ), there’s no way to sort without looking at the case. My words that start with a capitalize letter, will always be first. The order looks like this : “empty”, “space”, “number”, “uppercase”, “lowercase”.

The solution

You have 2 choices at the moment. Put everything lowercase or store 2 fields with the same value (original+lower for example). In my case, i used the “all lowercase” scenario.

There’s maybe another solution. It would be to create a full text index using an “/i/” options. This need to be tested

You can find more information here and here

p.s: 3.2 is out (1 month ago), but i’m pretty sure the issue is still there