Multithreading in php

Posted on Friday, January 2nd, 2009 under extreme php, multithreading, php

Hello there! If you are new here, you might want to subscribe to the RSS feed for updates on this topic.

Some time ago, I’ve wrote a small server in PHP. Nothing fancy. It would listen on a socket and when a new client would connect, the server would start a new thread and manage the client’s request. Since threading it’s not available in PHP, I’ve emulated the threads with child processes which are available in php. A thread object simply encapsulates a new process started with pnctl_fork() and emulates – to some extent – the behaviour of the java.lang.Thread class, the main difference being that in my implementation, you don’t extend the Thread class, you simply provide the name of a callback function in the constructor.

A simple multithreaded application would look like this:

require_once( 'Thread.php' );
 
// test to see if threading is available
if( ! Thread::available() ) {
	die( 'Threads not supported' );
}
 
// function to be ran on separate threads
function paralel( $_limit, $_name ) {
	for ( $index = 0; $index < $_limit; $index++ ) {
		echo 'Now running thread ' . $_name . PHP_EOL;
		sleep( 1 );
	}
}
 
// create 2 thread objects
$t1 = new Thread( 'paralel' );
$t2 = new Thread( 'paralel' );
 
// start them
$t1->start( 10, 't1' );
$t2->start( 10, 't2' );
 
// keep the program running until the threads finish
while( $t1->isAlive() && $t2->isAlive() ) {
 
}

This will display Now running thread 1 and Now running thread 2 messages with 1 second delays. I know, not that impressive, but hey, it’s multithreaded.

PHP threading will only work on Unix and Linux systems, because pnctl_fork is nothing more than a wrapper for the fork() function from unistd.h and it’s not available under Microsoft operating systems.

I know that the first example was pretty lame, but there are some more interesting things you could do with threads in PHP. For instance, if you need to do some server side processing of all the images in a directory, a multithreaded approach will be much faster.

require_once( 'Thread.php' );
 
// test to see if threading is available
if( ! Thread::available() ) {
	die( 'Threads not supported' );
}
 
// define the function to be run as a separate thread
function processImage( $_image ) {
	// expensive image processing logic here
}
 
$threads = array();
$index = 0;
 
foreach( new DirectoryIterator( '/path/to/images' ) as $item ) {
	if( $item->isFile() ) {
		$threads[$index] = new Thread( 'processImage' );
		$threads[$index]->start( $item->getPathname() );
		++$index;
        }
}
 
// wait for all the threads to finish
while( !empty( $threads ) ) {
	foreach( $threads as $index => $thread ) {
		if( ! $thread->isAlive() ) {
			unset( $threads[$index] );
		}
	}
	// let the CPU do its work
	sleep( 1 );
}

PS: it’s a bad practice to keep looping in order to wait for a thread to finish. An ongoing empty loop will quickly boost your CPU ’s load to 100%. If you need your processor free (and you need it), simply send the current (looping) thread to sleep and let the others execute.

// wait for thread - bad approach (overloads the CPU)
while ( $thread->isAlive() ) {
}
 
// wait for thread - correct approach
while( $thread->isAlive() ) {
    sleep( 1 ); 
}

Download Thread.php

Here you go: Thread.php. Just click the link to download the class. If you have a better approach to this issue or think that the original class could be improved, don’t be shy and leave a comment below. Credits will be given.

23 Responses to “Multithreading in php”

Trackbacks (1)

Multithreading in php | PHP-Blog.com

Comments (22)

• Raul •

January 27, 2009

Thanks!
• Tudor •

January 27, 2009

Did you try it on a live project? Can we see it at work?
• Raul •

January 27, 2009

Unfortunatelly “live” , “at work” no because of M$ OS.
But i’ll use it for my scripts in Ubuntu!
• Tudor •

January 28, 2009

I’ve used it on a project, but it’s only available in intranet. But it works and that’s all that matters. If you have any suggestions of improvement, so share
• upsemshop •

December 17, 2009

Hi guys,
My PC worked not correctly, too much mistakes and buggs. Please, help me to fix buggs on my computer.
My operation system is Win Vista.
Thx,
upsemshop
• Joey •

February 1, 2010

At line 170, there is an error.

pcntl_waitpid( $this->pid, $status = 0 );

You never use the variable $status anyway, so it doesn’t hurt. But if you ever decide to use the $status variable, passing in $status = 0 does not pass the variable to the function, it passes the value of the assignment, 0.

If you set your error reporting to strict, it will complain that:

“Only variables should be passed by reference”

Otherwise, thanks for sharing your code.
• Tudor •

February 1, 2010

10x man, I’ll look into it and fix it later today. Sometime bugs escape one programmer just to be caught by another. That’s the beauty of Open Source.
• Tudor •
February 7, 2010
I had a look at the code and it seems it was better the way it was. I made some tests, and, as if turned out, if I change that line and remove the $status variable, like such:
pcntl_waitpid( $this->pid, 0);
it will yield an fatal error, saying:
//Fatal error: Only variables can be passed by reference in Thread.php on line 170
I always want to pass in 0 as an argument there, so there are two ways in which that can be achieved:
// 1. The long way $status = 0; pcntl_waitpid( $this->pid, $status); // 2. The short way pcntl_waitpid( $this->pid, $status = 0);
Hope it’s clear for everybody now.
• Joey •

February 10, 2010

Why do you want to pass a zero in there? That parameter is not expecting a _value_, it’s expecting a _reference_. PHP can can put the real status value in it and give it back to you so that you can check against it later.

See the functions pcntl_wif*()

I hope that made sense.
• Tudor •

February 10, 2010

I’ve browsed the manual, as this entry is over one year old and I can’t remember why I took some decisions back then. It seems that you must always pass in a variable for the status to the function, in order to work. I’m not interested in the returned status, so I’ve passed in a zero variable.
• George •

March 30, 2010

Thanks for the example code. Any idea how I would use that in an Apache server that executes PHP scripts? I’m on Ubuntu and it seems as if pcntl_fork() is not available in the PHP that is in the apache. I tried the solution in

http://www.php.net/manual/en/ref.pcntl.php#91224

but it does not seem to work for me.

Any ideas?

Thanks,

George
• Tudor •

March 31, 2010

It won’t work like this. I think the best way to achieve multithreading is dropping PHP and switch to python or something else. Or rewrite in a different language the part you want to be multithreaded.
• Zoli •

April 8, 2010

Hi,

The function I’d like to thread returns a string, is it possible to retrieve that somehow and let them set in a result array in the main code starting the threads for example?

Thanks in advance, Zoli
• Tudor •
April 9, 2010
Good one, Zoli! I’ll update the Thread.php file in the future. Until then, you need to do the following: add “return” in front of the call_user_func_array() and
call_user_func() calls so that the start() method will return the original function’s result. Like such:
// child pcntl_signal( SIGTERM, array( $this, 'signalHandler' ) ); $arguments = func_get_args(); if ( !empty( $arguments ) ) { return call_user_func_array( $this->runnable, $arguments ); } else { return call_user_func( $this->runnable ); } exit( 0 );
This is untested code, but it *should* work!
• sims •

April 29, 2010

Thanks for that. I learned a lot from your code. A suggestion to an otherwise excellent class:

It is allowed for the first parameter of call_user_func_array to be an array(class, ‘method’). So, there is a problem with function_exists in runnableOk. Besides, if it is_callable then function_exists. So I don’t think you need to check both.
• Ricardo Sanchez •

May 5, 2010

Hi! Great script!

I’ve been testing the Zoli idea and the “return” aproximation and it does not work… mainly because the child dies and there is no communication between parent and child…

They are two separated processes, so… any idea to get returned data to the parent from the children function? Maybe sockets??

• Ricardo Sanchez •

May 6, 2010

Hi! I’ve found how to return data from child to parent, using pipes:

In class Thread:

	/**
	 * Starts the thread, all the parameters are passed to the callback function
	 * @return void
	 */
	public function start() {
		$status = 0;
                $results = null;
		$pid = pcntl_fork();
 
		if( $pid == -1 ) { //error forking, no child is created
			throw new Exception( $this-&gt;getError( Thread::COULD_NOT_FORK ), Thread::COULD_NOT_FORK );
		}else if ( $pid ) {// parent
			$this-&gt;pid = $pid; 
 
		} else { // child 
			$this-&gt;pid = posix_getpid();//pid (child)
			$this-&gt;ppid = posix_getppid();//pid (parent)
 
			pcntl_signal( SIGTERM, array( $this, 'signalHandler' ) );
			$array_args = func_get_args();
			if ( !empty( $array_args ) ) {
				$results = call_user_func_array( $this-&gt;runnable, $array_args );
			}else{
				$results = call_user_func( $this-&gt;runnable );
			}
 
			$pipe = "/tmp/pipe_"..$this-&gt;pid;//pid is known by parent
 
			if(!file_exists($pipe)) {//child talks to parent using this pipe
				umask(0);
				posix_mkfifo($pipe, 0600);
			}
			//we have to open the pipe and send the data serialized
			$pipe_descriptor = fopen($pipe, 'w');
			fwrite($pipe_descriptor, serialize( $results ) );
 
			//and kill the child using posix_kill ( exit(0) duplicates headers!! )
			posix_kill( $this-&gt;pid , SIGKILL);
			exit(0);
		}
	}

In function which uses Thread:

$sleep_microseconds = 200000; // we don't want to achieve 100% CPU load!
 
//start multiple threads!!
for ($i = 0; $i start( 'param1','param2' );
	}
}
 
//We are the parent... childs never reach this part of the code!!
//Parent has to read child's pipes... Communication between parents and childrens is necesary in our society!
 
while (!empty( $threads )){//while any child is alive...
	foreach($threads as $index =&gt; $thread){
		$child_pipe = "/tmp/pipe_".$thread-&gt;getPid();//this pipe communicate the parent with one of his childs... look at Thread-&gt;start() method, in both places (child and parent) the pipe name must be the same!
 
		if (file_exists($child_pipe)){//if pipe exists, we (the parent) open it, listen to our child, and when the child has finished, the child will die and the parent close the pipe between them
			$file_descriptor = fopen( $child_pipe, "r");
			$child_response = "";
			while (!feof($file_descriptor)){//while pipe is open, parent continues reading
				$child_response .= fread($file_descriptor, 8192);
			}
			//we have the child data in the parent, but serialized:
			$array_responses[ $index ] = unserialize( $child_response );
 
			//now, child is dead, and parent close the pipe
			unlink( $child_pipe );
			unset($threads[$index]);//and remove the child from our array
		}
	}
	//parent must sleep for a while, because we don't want to achieve 100% CPU load!
	usleep( $sleep_microseconds );
}
 
//now, we have all data from the forked childs in $array_responses

• Shane Mc Cormack •

July 15, 2010

I needed to pass a value by reference to the “threaded” function and noticed that in its current instance, this class only passes by ref, this was changed near line 147 by using:

            $stack = debug_backtrace();
            $arguments = array();
            if (isset($stack[0]["args"])) {
                for($i=0; $i > count($stack[0]['args']); $i++) {
                    $arguments[$i] = & $stack[0]['args'][$i];
                }
            }

I’m not sure of the performance hit by doing this, but it works.

• Shane Mc Cormack •

July 15, 2010

Further to the above, I was mistaken, that obviously doesn’t work, all it does is stop the warnings.

It only appeared to be working based on the input I was testing with, oh well.
• Tudor •

July 15, 2010

Encapsulate the value you want to pass on in an object. Since objects are always passed by reference in PHP, it should do the trick.
• Jelle De Loecker •

August 21, 2010

The threading itself worked like a charm, unfortunately my singleton mysql wrapper wouldn’t work with it as it always complained about the database going away.
• Tudor •

August 24, 2010

Try opening a permanent connection to the MySQL server!

most commented posts

recent comments

tag cloud

programming blogs

friends blogs

non-english blogs

Multithreading in php

Download Thread.php

Related posts

23 Responses to “Multithreading in php”

Trackbacks (1)

Comments (22)

Leave a Reply