Kris Wallsmith

How to Spam Twitter in 3 Easy Steps (or, The Death of Twitter)

Despite the fleeting “spammers perish” event a few days ago, my Twitter profile is still overrun by spammy followers. This is really bugging me. I’ve been forced to switch notifications off, and my stop_jackin_it.php script isn’t working because the /blocks/create API method is broken. This will be the downfall of Twitter if it isn’t contained.

So you want to be a spammer too?

I’m going to explain how I would replicate this attack in an attempt to move this conversation forward. I’ve spent way too much time thinking about this and writing this article, considering I don’t get a dime from Twitter. It’s my user loyalty that has me putting this time in, but that won’t last too much longer.

Anyway, here’s how I would reproduce what’s going on:

1. Create a bunch of fake accounts and have them follow each other so they look credible. About 200-300 followers should do the trick. Twitter has a captcha in place, so the signup process will require human eyes, but you should be able to handle this by having just a handful of people create 10-20 accounts a day. If you’re a gung-ho spammer, you could also come up with a crowd-sourcing scheme to get the job done.
2. Scrape tweets from the public timeline into a database. Keep track of when each tweet was created though — you’ll want to wait at least 3 weeks before replaying so the source tweet is expunged from Twitter’s search index.
3. Once you’ve held onto a tweet for 3 weeks, have about half-a-dozen of your spam soldiers replay this tweet. Have one or two of them include a link to your porn site, minified by, say, xurl [dot] jp. Here are some examples.

That’s it!

What is Twitter doing about this?

Twitter doesn’t seem to be doing much about this. I lost some followers to “spammers perish,” but they were quickly replaced by many more. If Twitter wants to beat this, I suggest they do one of the following:

1. Fix the API! Goodness gracious, why doesn’t /blocks/create work yet? An ambitious developer could get a really nice spam blocking SaaS up and running pretty easily if this method actually worked.
2. Delete these accounts for us. This ongoing attack is using xurl [dot] jp exclusively, for some reason, so this should be pretty easy. You could even take advantage of shortening service’s API to confirm abuse.
3. Hire me to do it.

I hope this can be resolved soon. I’m losing interest in Twitter by the day.

Doctrine and MySQL integers

I just pasted this somewhere handy for my own reference. It’s the logic Doctrine uses to translate the integer data types you specify in schema.yml into a MySQL data type. If you’ve ever wondered why the length you set on an integer isn’t directly translated to the table definition used in your database, here’s why.

For example, the id columns in sfDoctrineGuardPlugin are defined as integer(4), which translates to id INT in the table definition. Where did the 4 go? Now you know…

stop_jackin_it.php

I’ve been getting inundated by followers who appear to be normal people with a healthy number of followers, but have links to xurl.jp spiced throughout their timeline which resolve to… wait for it… porn.

I’m sick of it. So I did something about it. I cronned this script on my computer and you should to. It will run a search for the string xurl.jp and block anyone posting links to this site. It will only deal with this particular spammer, but it’s better than doing this manually.

The best part about it is the blocks.create method isn’t rate-limited. Thanks Twitter!

Hopefully you won’t lose too many followers! I think I lost about 10…

Important!

Please don’t include the text xurl.jp in any tweets about this post, or my script will block you too!

Symfony: Denote Required Form Fields

You can checkout the full gist for this tutorial here.

The symfony form framework separates a form’s presentation and validation into two distinct collections of classes: widgets and validators. For the most part, these two codebases live happily without any knowledge of eachother. When data from the validators needs to be shown in the presentation layer, such as in the case of error messages, symfony provides the sfFormField class, which bridges this divide.

One common requirement for web forms is that required fields must be apparent to the user. Implementing this in the symfony form framework may not seem possible, because of the divide between widgets and validators, but it is in fact quite possible, and easily implemented.

This is a quick tutorial on how to do just that.

Validator, meet Widget

The first step is for the form’s validators to inform the form’s widgets which of its fields are required. We can do this by passing a array of field names to the widget schema once the form is fully configured:

The getRequiredFields() method returns a simple array of formatted field names that corresponds to those validators marked as required. It accomplishes this by recurrisively iterating through the form’s validators:

Label Formats

Once the form’s widgets know which fields are required, it’s up to the form formatter to decorate those fields accordingly. I’ve done this using an extension of the standard table formatter class:

Finally, we just add this custom formatter to our forms by adding the following to the overloaded __construct() method from above:

Your forms’ required fields should now render with labels that include the required class. Of course, you can do something different in the formatter to suit your needs, add a * or add a class to the entire row, but hopefully the implementation should now be clear.

MooTools: Bubbling Controllers

This one’s an oldie but goodie I just pulled out of my tome of a ~/Sites directory. Similar to Aaron Newton’s concept of Events Arbiters, this class fires events for one object on another object, but I’ve applied the bubbling pattern you may be familiar with from native DOM events. Instead of bubbling up a hierarchy of elements, we just bubble up a hierarchy of controllers.

Here’s the class:

Events.BubblesTo = new Class({

  Extends: Events,

  bubblesTo: function(parent, namespace) {
    this.$bubblerParent = parent;
    this.$bubblerNamespace = namespace;
    return this;
  },

  fireEvent: function(type, args, delay) {
    this.parent(type, args, delay);

    if (this.$bubblerParent &&
      'function' == $type(this.$bubblerParent.fireEvent)) {
      if (this.$bubblerNamespace && !type.contains(':'))
        type = this.$bubblerNamespace+':'+type;
      this.$bubblerParent.fireEvent(type, [this].concat(args));
    }

    return this;
  }

});

Here’s a sample implementation:

var Dealer = new Class({
  Implements: Events,
  initialize: function(name){
    this.name = name;
    this.addEvents({
      'deck:shuffle': function(deck){
        alert(this.name+' shuffled deck '+deck.nb);
      },
      'card:flip': function(card, deck){
        alert(this.name+' flipped a '+card.name+' from deck '+deck.nb);
      }
    });
  }
});

var DeckOfCards = new Class({
  Implements: Events.BubblesTo,
  initialize: function(dealer){
    this.nb = ++DeckOfCards.counter;
    this.bubblesTo(dealer, 'deck');
  },
  shuffle: function(){
    this.fireEvent('shuffle', this);
  }
});
DeckOfCards.counter = 0;

var PlayingCard = new Class({
  Implements: Events.BubblesTo,
  initialize: function(deck){
    this.bubblesTo(deck, 'card');
  },
  flip: function(){
    this.fireEvent('flip', this);
  }
});

var dealer = new Dealer('John');
var deck = new DeckOfCards(dealer);
var cards = ['king', 'queen', 'jack'].map(function(name){
  return new PlayingCard(deck);
});

deck.shuffle();
// --> alerts "John shuffled deck 1"

cards.getRandom().flip();
// --> alerts "John just flipped a ___ from deck 1"

Notice the events are both fired from one controller and listened to on another controller. With Events.BubblesTo in place, the Dealer class can easily keep a watchful eye over its decks and cards without having to listen to each one individually.

Sorry for the lack of syntax highlighting, BTW. Anyone have a good way to accomplish PHP and Javascript syntax highlighting on Tumblr?

MooTools: Ignoring the next click

I’m a big fan of MooTools, as I recently tweeted. I typically try to extend this framework as little as possible, since that can be rabbit hole for me, but this method is just too handy to pass by.

The use case I’m working with is distinguishing between a drag/drop interaction and a click interaction on the same element. If you’re dragging an element, the click event will be fired when you drop it, but you may not want to execute the typical click behavior.

Native.implement([Element, Window, Document], {
  ignoreNext: function(type){
    var element = this, events = element.retrieve('events', {});
    var functions = events[type] || { keys: [], values: [] };
    element.removeEvents(type).addEvent(type, function(event){
      event.stop();
      element.removeEvents(type);
      functions.keys.each(function(fn){ element.addEvent(type, fn) });
    });
  }
});

I have this implemented in a drag start handler like so:

element.addEvent('click', function(){ alert('click') }).makeDraggable({
  onStart: function(drag){ drag.ignoreNext('click') },
  onDrop: function(){ alert('drop') }
});

Assuming the Moo team doesn’t change how native events are handled, this gives you a nice reusable method for handling this otherwise brainfart-inducing interaction.

Managing Master and Slave Database Connections with symfony and Doctrine

This is one of those things that should be much easier than it is. Since I started using Doctrine a few months ago, I’ve been impressed with how complete it is, but I can definitely see room for improvement as the project matures. Setting up read and write connections is one of those areas.

My challenge was to get the project I’m on ready to be hosted on Amazon EC2, with the help of RightScale. If your hosting on the cloud and have the funds available, check out RightScale. They have a number of server templates with best practices already implemented, and great online tutorials.

The environment I’m setting up includes one master and (potentially) multiple slave MySQL servers (setup with EBS storage). Setting up these servers was a piece of cake, thanks to the heavy lifting RightScale had done for me with their server templates and tutorials. The challenge I met was in getting symfony (1.2) and Doctrine (1.1) setup to choose the right connection for each query.

Code Slingin’ Below

The first order of business is organizing the database connections. I decided to go with a simple naming syntax and assume the master connection is named master and the slave connection names begin with slave. I added these methods to ProjectConfiguration so these connections are accessible:

Now I can easily grab the master connection by calling ProjectConfiguration::getActive()->getMasterConnection() from anywhere in my project, and get a slave connection by calling ->getSlaveConnection(). This is cool, but it turned out to be the easiest part, by far.

Smartly Forcing a Master or Slave Connection

I started with a tutorial on master and slave connections in the Doctrine documentation repository. This was easy to implement, but it’s not a complete solution. There are write queries in the Doctrine core that don’t go though either Doctrine_Query::preQuery() or Doctrine_Record::save(), so they end up using the current connection, which is usually the slave connection.

I came up with a solution, but I’m witholding judgement on how stable it is. I’m using Doctrine events to filter all queries run through Doctrine and swap out the PDO object used inside the Doctrine_Connection object with either the master or slave PDO object:

Then simply add this listener to each of you Doctrine_Connection objects by using the (undocumented?) ProjectConfiguration::configureDoctrineConnection() method:

This seems to be working for my purposes, but I feel a bit dirty hacking into Doctrine_Connection objects like this. For a brief moment as each query is run, the Doctrine_Connection object that includes the name master may in fact include a slave PDO object, and vice versa.

One More Thing

This new environment includes multiple database servers, so it naturally includes multiple load-balanced web servers, which precludes the use of sfSessionStorage because of it’s reliance on a local filesystem. I decided to go with sfPDOSessionStorage for the time being.

I was expecting to have to extend sfPDOSessionStorage to choose between master and slave connections for each storage operation. However, upon looking at the code I realized that every operation may possibly include a write query, so I just specified the master connection in factories.yml:

NOTE This configuration assumes there is always a connection named master, whereas the method in ProjectConfiguration includes a fallback to the current Doctrine connection.

Doctrine Timestamps and User Timezones

I recently added a “timezone” dropdown to the user preferences screen on a symfony application currently in development. This simple extension to the sfDoctrineRecord class makes it easy to present times from the database in the current user’s timezone.

abstract class myRecord extends sfDoctrineRecord
{
  protected function _get($fieldName, $load = true)
  {
    if ($value = parent::_get($fieldName, $load))
    {
      $column = $this->getTable()->getColumnDefinition($fieldName);
      if ($column && 'timestamp' == $column['type'])
      {
        $timezone = date_default_timezone_get();
        if (ProjectConfiguration::DEFAULT_TIMEZONE != $timezone)
        {
          // shift value to the current timezone
          date_default_timezone_set(ProjectConfiguration::DEFAULT_TIMEZONE);
          $time = strtotime($value);

          date_default_timezone_set($timezone);
          $value = date('Y-m-d H:i:s', $time);
        }
      }
    }

    return $value;
  }

  protected function _set($fieldName, $value, $load = true)
  {
    $column = $this->getTable()->getColumnDefinition($fieldName);
    if ($column && 'timestamp' == $column['type'] && $time = strtotime($value))
    {
      $timezone = date_default_timezone_get();
      if (ProjectConfiguration::DEFAULT_TIMEZONE != $timezone)
      {
        // shift value to the default timezone
        date_default_timezone_set(ProjectConfiguration::DEFAULT_TIMEZONE);
        $value = date('Y-m-d H:i:s', $time);

        date_default_timezone_set($timezone);
      }
    }

    return parent::_set($fieldName, $value, $load);
  }
}

To get sfDoctrinePlugin to use this class instead of the default, sfDoctrineRecord, add the following to your ProjectConfiguration.

// config/ProjectConfiguration.class.php
class ProjectConfiguration extends sfProjectConfiguration
{
  public function setup()
  {
    // ...

    sfConfig::set('doctrine_model_builder_options', array(
      'baseClassName' => 'myRecord',
    ));
  }

  // ...
}

svn import: inconsistent newlines

I encountered this error ad nauseum while importing external libraries into my local/offline Subversion repositories:

File ‘/foo/bar’ has inconsistent newlines

At first I was opening each file in Textmate and then “save as”-ing with Unix newlines. This takes way too much time, so I wrote this little PHP diddy:

<?php

// Usage:
// php smart_svn_import.php . http://repo "Initial import"

$command = 'svn import --quiet %s %s -m %s 2>&1';
$command = vsprintf($command, array_map('escapeshellarg', array(
  $argv[1], // path
  $argv[2], // url
  $argv[3], // message
)));

while (true)
{
  ob_start();
  passthru($command, $return);
  $content = ob_get_contents();
  ob_end_clean();

  if (0 == $return)
  {
    break;
  }

  if (preg_match('/^svn: File \'(.*)\' has inconsistent newlines$/m', $content, $match))
  {
    // fix this file
    file_put_contents($match[1], str_replace(
      array("\n", "\r\n"),
      array(PHP_EOL, PHP_EOL),
      file_get_contents($match[1])
    ));
  }
  else
  {
    throw new Exception($content);
  }
}