Tarantool Queue alternatives and similar libraries
Based on the "Queue" category.
Alternatively, view Tarantool Queue alternatives based on common mentions on social networks and blogs.
InfluxDB - Power Real-Time Data Analytics at Scale
* Code Quality Rankings and insights are calculated and provided by Lumnify.
They vary from L1 to L5 with "L5" being the highest.
Do you think we are missing an alternative of Tarantool Queue or a related project?
README
Tarantool Queue
Tarantool is a NoSQL database running in a Lua application server. It integrates Lua modules, called LuaRocks. This package provides PHP bindings for Tarantool Queue LuaRock.
Table of contents
Installation
The recommended way to install the library is through Composer:
composer require tarantool/queue
Before start
In order to use queue, you first need to make sure that your Tarantool instance is configured, up and running. The minimal required configuration might look like this:
-- queues.lua
box.cfg {listen = 3301}
queue = require('queue')
queue.create_tube('foobar', 'fifottl', {if_not_exists = true})
You can read more about the box configuration in the official Tarantool documentation. More information on queue configuration can be found here.
To start the instance you need to copy (or symlink) queues.lua
file into the /etc/tarantool/instances.enabled
directory and run the following command:
sudo tarantoolctl start queues
Working with queue
Once you have your instance running, you can start by creating a queue object with the queue (tube) name you defined in the Lua script:
use Tarantool\Queue\Queue;
...
$queue = new Queue($client, 'foobar');
where $client
is an instance of Tarantool\Client\Client
from the tarantool/client package.
Data types
Under the hood Tarantool uses MessagePack binary format to serialize/deserialize data being stored in a queue. It can handle most of the PHP data types (except resources and closures) without any manual pre- or post-processing:
$queue->put('foo');
$queue->put(true);
$queue->put(42);
$queue->put(4.2);
$queue->put(['foo' => ['bar' => ['baz' => null]]]);
$queue->put(new MyObject());
To learn more about object serialization, please follow this link.
Tasks
Most of the [Queue API](src/Queue.php) methods return a [Task](src/Task.php) object containing the following getters:
Task::getId()
Task::getState() // States::READY, States::TAKEN, States::DONE, States::BURY or States::DELAYED
Task::getData()
And some sugar methods:
Task::isReady()
Task::isTaken()
Task::isDone()
Task::isBuried()
Task::isDelayed()
Producer API
As you've already seen, to insert a task into a queue you need to call put()
method, which accepts
two arguments: the data you want to process and optional array of task options, which this particular
queue supports. For example, fifottl
queue (which we defined earlier in our Lua config
file), supports delay
, ttl
, ttr
and pri
options:
use Tarantool\Queue\Options;
$queue->put('foo', [Options::DELAY => 30.0]);
$queue->put('bar', [Options::TTL => 5.0]);
$queue->put('baz', [Options::TTR => 10.0, Options::PRI => 42]);
See the full list of available options here.
Consumer API
To reserve a task for execution, call take()
method. It accepts an optional timeout
parameter.
If a timeout value is supplied the call will wait timeout
seconds until a READY
task appears in the queue.
The method returns either a Task object or null
:
$taskOrNull = $queue->take();
// wait 2 seconds
$taskOrNull = $queue->take(2.0);
// wait 100 milliseconds
$taskOrNull = $queue->take(.1);
After successful execution, a task can be marked as acknowledged (that will also delete the task from a queue):
$data = $task->getData();
// process $data
$task = $queue->ack($task->getId());
Or put back into the queue in case it cannot be executed:
$task = $queue->release($task->getId());
// for *ttl queues you can specify a delay
$task = $queue->release($task->getId(), [Options::DELAY => 30.0]);
To look at a task without changing its state, use:
$task = $queue->peek($task->getId());
To bury (disable) a task:
$task = $queue->bury($task->getId());
To reset buried task(s) back to READY
state:
$count = $queue->kick(3); // kick 3 buried tasks
To increase TTR and/or TTL of a running task (only for *ttl queues):
$taskOrNull = $queue->touch($takenTask->getId(), 5.0); // increase ttr/ttl to 5 seconds
A task (in any state) can be deleted permanently with delete()
:
$task = $queue->delete($task->getId());
To delete all tasks in a queue:
$queue->truncate();
For a detailed API documentation, please read the section "Using the queue module" of the queue README.
Statistics
The stats()
method provides access to the statistical information accumulated
since a queue was created:
$stats = $queue->stats();
The result of this call might look like this:
[
'tasks' => [
'taken' => 1,
'buried' => 1,
'ready' => 1,
'done' => 0,
'delayed' => 0,
'total' => 3,
],
'calls' => [
'bury' => 1,
'put' => 3,
'take' => 1,
...
],
]
In addition, you can specify a key to return only a subset of the array:
$calls = $queue->stats('calls');
$total = $queue->stats('tasks.total');
Custom methods
Thanks to flexible nature of the queue Lua module, you can easily create
your own queue drivers or extend existing ones with an additional functionality. For example, suppose you added
the put_many
method to your foobar
queue, which inserts multiple tasks atomically:
-- queues.lua
...
queue.tube.foobar.put_many = function(self, items)
local put = {}
box.begin()
for k, item in pairs(items) do
put[k] = tube:put(unpack(item))
end
box.commit()
return put
end
To invoke this method from php, use Queue::call()
:
$result = $queue->call('put_many', [
'foo' => ['foo', [Options::DELAY => 30.0]],
'bar' => ['bar'],
]);
Testing
The easiest way to run tests is with Docker. First, build an image using the [dockerfile.sh](dockerfile.sh) generator:
./dockerfile.sh | docker build -t queue -
Then run a Tarantool instance (needed for integration tests):
docker network create tarantool-php
docker run -d --net=tarantool-php -p 3301:3301 --name=tarantool \
-v $(pwd)/tests/Integration/queues.lua:/queues.lua \
tarantool/tarantool:2 tarantool /queues.lua
And then run both unit and integration tests:
docker run --rm --net=tarantool-php -v `pwd`:/queue -w /queue queue
To run only integration or unit tests, set the PHPUNIT_OPTS
environment variable
to either --testsuite integration
or --testsuite unit
respectively, e.g.:
docker run --rm --net=tarantool-php -v `pwd`:/queue -w /queue \
-e PHPUNIT_OPTS='--testsuite unit' queue
License
The library is released under the MIT License. See the bundled [LICENSE](LICENSE) file for details.
*Note that all licence references and agreements mentioned in the Tarantool Queue README section above
are relevant to that project's source code only.