Anonymous Subs and Reference Counters

Monday, January 02, 2006, at 03:04PM

By Eric Richardson

I apologize to those of you who have no idea what I'm taking about in these coding posts. Just skip over them and you won't miss anything.

I had a breakthrough today in the whole memory leak issue when I realized that the way I am doing lazy initialization of some objects that I may or may not need but want available. The troublesome code:

# create system object
$swb->register('system',sub {
    $self->_->new_object('System');
});

As long as that anonymous sub stays in existence, $self is sitting there as a reference and the calling object isn't going to fall out of scope.

The Setup

I'm not sure if any given run is going to need a 'System' object. I don't want to initialize it if I don't have to, but I want creation to be absolutely transparent to the caller. The way I've been doing it is to pass a CODEREF into the switchboard (here's a link to the Switchboard code). If the created accessor is called and sees that it has a CODEREF it'll run it and return the results. So, for instance, when it runs the anonymous sub above it'll get an eThreads::Object::System object and return (and store) that. Later calls would get the same object.

The Problem

The problem is that the anonymous sub I'm passing in isn't mod_perl-safe. Since it includes that call to $self it keeps a reference lying around, and if the lazy accessor is never initialized that reference never falls out of scope and therefore trips up Perl's reference counter. If you do use the accessor there's no problem, but that doesn't help the real problem.

Complicating matters is that the anonymous sub does need a reference to its creating context. When executed it's inside the Switchboard::Accessors package, so without storing a ref things like $self wouldn't do what you want them to do.

What to Do...

I guess in an ideal situation you'd want a way to weaken any references inside the anonymous sub. You'd also want a way to see if any of the references fall out of existence while you're holding on to the sub.

How do you do that? Well... That's a good question.

Internally Perl knows the lexical vars contained inside the anonymous sub. If you use Devel::Peek to dump the coderef you can see them yourself yourself in the scratchpad (PADNAME). You can even get at the pad yourself:

my $sub = sub () {
    $self->foo
};

my $cv = bless \$sub , 'B::CV';
my ($names,$values) = ($cv->PADLIST->ARRAY);
my @names = $names->ARRAY;
my @values = $values->ARRAY; 

for (my $i = 0; $i < @names; $i++) {
    next if ( ref($names[$i]) eq "B::SPECIAL" );
    printf STDERR 
        "%d: %s = %s\n",
        $i,
        $names[$i]->terse,
        $values[$i]->terse;
}

But that's as far as I've gotten. I can see that the ref I get from the pad points to the same memory address as $self, but I haven't successfully weakened the reference count yet. I haven't thought through the idea of just decrementing the counter on $self enough to know whether it would be smart or not.

So now I know exactly where the leak is, and I've got a small test app to work with. All that's left is a solution.