Tweekly.fm Moves to MongoDB

Posted on: Friday, 29th October 2010

Over the past two days I’ve been making a lot of changes to how tweekly.fm operates and functions for both users and myself. We no longer user MySQL for the majority of the site because concurrently, it just couldn’t handle what we were throwing at it very well. By moving over to a NoSQL alternative (in our case, MongoDB) we can now support a lot more users concurrently and processing over 35,000 updates on a Sunday no longer brings the service down the a slugs pace.

One of the main problems that I had was debugging when users had problems with the service. During the move over to MongoDB I’ve added a whole host of logging integration points to detect when an interaction succeeds or fails. It will now be logged to our logging store and the most recent log response will be stored in your user record and visible on your settings page. This will help identify when something does go wrong, where and why it does.

I can’t stress how important it is that you confirm your Last.fm account too. When you go through this authorisation process I get a token back to access your play data on Last.fm which I can then user to pull your data even when your account is protected. Its also a lot more failsafe than just having a legacy Last.fm username set - although for the past two months its been mandatory. There was also a slight bug that happened with a subset of protected accounts which has now been fixed too. I’m quite confident that your play data will be pulled correctly every single time. You can manually update your accounts play data by visiting the ‘Publish Now’. You don’t need to actually push an update out, just visiting the page will update your account.

The next change affects your personal tweekly.fm URL. Twitter users will now be found at http://www.tweekly.fm/twitter/dordotky and Facebook users will get their own URL at http://www.tweekly.fm/facebook/501640297 which functions in the exact same way that the original tweekly.fm user pages. Facebook users will also get a short bit.ly URL. Comments will return to these pages shortly.

The Facebook issue of cutting off updates has also been fixed with a new version of the processing engine that builds updates for users. You shouldn’t see an errors like that again. Shared play counts, artist data and comments will return within a few days. If you notice any problems or something doesn’t work as expected then please do let me know.