Twitterfall 1.5: What’s new?

Today I launched a ‘new’ version of Twitterfall, but everything looks pretty much the same so what is actually new?


That translucency seemed like a great idea at the time.


Twitterfall is over 6 years old now! In that time, the Internet (and Twitter) have come a long way, but Twitterfall has mostly stayed the same. People are fond of Twitterfall as it is, so I don’t want to mess with that too much, but there are some aspects that have eroded away over time, which I have now finally set right.

Some of this post will be technical. If you’re not technically minded,¬†gloss over it ūüôā

JavaScript has come a long way…

The code that powers Twitterfall hasn’t changed much in about 4 years (!), which means that despite the new version looking the same a lot of the code has been completely reworked and cleaned up. In particular 6 years ago browsers were much less performant than they are now. Back then, we were at the forefront of what browsers could do, but now Twitterfall is pretty easy-going for your web browser in 2015. As such a lot of code designed to make Twitterfall as fast as possible back in 2009 is now irrelevant and could be stripped away to help make it more maintainable. Hopefully in the future I can actually add features and fix things!

Originally, the JavaScript powering Twitterfall was one single 3300 line JavaScript file (called ‘twit.js’ for no reason). With new build tools like Grunt and Browserify I have been able to split this up into 11 different files, and that’s just the start¬†(the main file is still 2700 lines); this paves the way for splitting it up even more to make it much more maintainable.

…so has the Twitter API…

The Twitter API has undergone massive changes since Twitterfall first launched. Did you know that when Twitterfall first came out, Twitter had no oAuth? You had to trust us with your password. I’m glad those days are over. But even since then, unauthenticated API calls have changed, the search API has been brought in line with the REST API, and Twitter added Entities to tweets.

Therefore, a lot of ‘Twitterfall 1.5’ was about fixing Twitterfall to make use of new API features.

Perl, PHP, Erlang, now Go

The backend for Twitterfall is mostly just a proxy to Twitter. But with lots of requests coming through it needs to be reasonably performant. At first we used Perl, but that was terrible. It was rewritten in PHP, then Erlang. Erlang suited well but was heavy on the CPU, so the proxy has now been rewritten in Go. This was largely because I wanted to try Go; JavaScript or Ruby would’ve also been a good choice. So far it is performing well, but time will tell.

Key changes

  • Grunt-based build system.
  • No need to construct fancy data structures like Tries when browsers can iterate over arrays hundreds of times faster than they used to.
  • String concatenation is not a bottleneck; no need for two different methods of concatenating strings that was chosen based on the user’s browser.
  • Templating with ICanHaz and Mustache has replaced raw HTML construction in JS strings.
  • Falling back to cookies if LocalStorage is unavailable isn’t necessary when LocalStorage is everywhere. Additionally, this means LocalStorage can actually be used properly.
  • Tweet entities mean Twitterfall doesn’t need¬†to parse tweet text by hand to look for¬†URLs, images, etc.
  • No one uses yfrog, mobypicture or¬†plixi or twitgoo (or others!) for sharing photos on Twitter any more, but Twitterfall still supported them all. A lot of code to handle different photo sharing services has¬†be removed.
  • People are happy with ‘new style’ retweets now, so Twitterfall now¬†supports them by rendering retweets in a similar style to¬†the official Web client.
  • The Search API and the¬†REST API provide identical Tweet objects now, so no need for all the repeated code handling the subtle differences.
  • Caching global variables locally inside¬†functions is no longer worth the¬†pain, so it could be removed
  • Caching functions inside functions that call them to reduce global variable space is no longer¬†relevant, so it could be removed.

What next?

Still true to how it used to look.

Still true to how it used to look, with a few less options.

I’d like to start gathering¬†some more information about how people use Twitterfall so I can make some¬†small changes to improve it. In general I don’t want to change anything too drastic, though I do want to update the colour scheme! There’s also still a few outstanding bugs.

Over time, I’d like to also migrate isolated parts of the code to¬†newer technologies like React, ES6 and Backbone.¬†Again, to help¬†fixing bugs and adding features easier.

And finally, I’d love to make Twitterfall realtime with Site¬†Streams. I’ve¬†made several proof-of-concepts over the years. Perhaps with this refreshed, more maintainable code, it might become a reality.

As always, thank you for continuing to use Twitterfall, and let me know if you have any problems or requests!

This entry was posted in Blog, Noteworthy, Twitterfall. Bookmark the permalink.