Under development — the help is being filled in plugin by plugin.
Help/Plugin reference/queue
Plugin reference

queue

A FIFO queue for passing values between producers and consumers.

14 functionsnamespace queuesource plugins/queue/QueuePlugin.c

Written from a line-by-line source review; every example output is from a real run.

Introduction

The queue plugin is a named, in-memory FIFO (first-in, first-out) message queue. Several independent queues can exist at once, each identified by a unique name. You put items in (Push) and take them out in the order they were added (Pop). Items come in two kinds: text (string) or binary data (blob). The queue is thread-safe: several threads can safely put in and take out items at the same time, which makes it a typical tool for the producer-consumer pattern.

Blocking and non-blocking operations

Every operation comes in two flavors. The plain Push and Pop do NOT block: Push returns 0 at once if the queue is full, and Pop returns an empty value if the queue is empty. The Wait variants (WaitPush, WaitPop and their blob counterparts) WAIT instead: WaitPop goes to sleep (0% CPU) until another thread puts in an item, and wakes immediately. This is what enables efficient, spin-free work in background threads.

Timeout

The last argument of the Wait variants is the time limit in milliseconds. If timeoutMs > 0, the function waits at most that long, then returns unsuccessfully (0, or an empty value). If timeoutMs is 0 or negative, the wait is INFINITE (until the item can be inserted or taken).

Loading the plugin

plugin "../plugins/print/PrintPlugin";
plugin "../plugins/queue/QueuePlugin";

A typical flow — producer and consumer

queue.Create("jobs", 100);
queue.Push("jobs", "task-1");
// on another thread:
printf("%s\n", queue.WaitPop("jobs", 0)); // infinite wait
queue.Destroy("jobs");

What to know about every function (the basics)

  • FIFO order: items come out in exactly the order you put them in.

  • Items are COPIED. Push copies the source string or blob at the moment of insertion, so you may modify it freely afterwards; Pop returns a fresh copy.

  • Using a non-existent queue is an ERROR. Unlike the map plugin, Push, Pop, Size and so on raise a runtime error for a non-existent name and the script stops — so create the queue first with Create (or check with Exists).

  • Create is also an error if the name is already taken, empty, or too long. Only Exists/Destroy tolerate a non-existent queue (returning 0).

  • Type purity: a string Pop can only take a string item. If the queue holds a blob, Pop raises an error — use PopBlob then (and vice versa). It is best to keep a single item kind per queue.

  • Unbounded queue: if the Create maxItems parameter is 0 (or negative), the queue has no upper size limit, and Push never returns full (0).

How to read the signatures

name is always the queue's name. The type after the -> arrow is the return type. ref blob $Item marks that the blob must be passed by reference (the $-prefixed variable directly). For example, queue.WaitPop(name, timeoutMs) -> string takes two arguments and returns a string.

Lifecycle

Creating, destroying, and checking the existence of a queue.

queue.Create

queue.Create(name, maxItems) -> int

Creates a new, empty FIFO queue with the given name and maximum item count.

Parameters
Parameter Type Description
name string The unique name of the queue. May not be empty, and has a built-in length limit.
maxItems int The maximum number of items. 0 or negative means an unbounded queue.
Return value

1 if it was created. If the name is already taken, empty, or too long, a runtime error.

Example
printf("%d\n", queue.Create("q", 3));
printf("%d\n", queue.Capacity("q"));
queue.Destroy("q");
Output after running
1
3
When to use

Create the queues you need at the start of the program. Use maxItems to cap memory use (backpressure): on a full queue, Push rejects and WaitPush waits until room frees up.

queue.Destroy

queue.Destroy(name) -> int

Destroys the queue, frees all remaining items, and wakes any threads waiting on it.

Parameters
Parameter Type Description
name string The name of the queue to destroy.
Return value

1 if it was destroyed; 0 if there was no queue with that name.

Example
queue.Create("q", 3);
printf("%d\n", queue.Destroy("q"));
printf("%d\n", queue.Destroy("q"));
Output after running
1
0
When to use

Call it when the work is done. Important: stop the background threads using the queue first, because Destroy does not wait for pending Wait calls — it wakes them, and they return unsuccessfully.

queue.Exists

queue.Exists(name) -> int

Tells whether a queue with the given name exists.

Parameters
Parameter Type Description
name string The name of the queue to check.
Return value

1 if it exists; 0 if not.

Example
queue.Create("q", 3);
printf("%d\n", queue.Exists("q"));
printf("%d\n", queue.Exists("nope"));
queue.Destroy("q");
Output after running
1
0
When to use

Since Push/Pop raise an error on a non-existent queue, you can cautiously check with Exists before using a queue of uncertain origin.

Inserting (producer side)

Adding an item to the end of the queue. Push is non-blocking; WaitPush waits on a full queue until room frees up.

queue.Push

queue.Push(name, item) -> int

Puts a text item at the end of the queue. Non-blocking: if the queue is full, it returns unsuccessfully at once.

Parameters
Parameter Type Description
name string The name of the queue.
item string The text to insert (it is copied).
Return value

1 if it was inserted; 0 if the queue was full. A non-existent queue causes a runtime error.

Example
queue.Create("q", 3);
printf("%d\n", queue.Push("q", "a"));
queue.Push("q", "b");
queue.Push("q", "c");
// the queue is now full (3/3)
printf("%d\n", queue.Push("q", "d"));
queue.Destroy("q");
Output after running
1
0
When to use

Good when you do not want to wait: the 0 return tells you the queue is full, and you can handle it differently (drop the item, or try later).

queue.WaitPush

queue.WaitPush(name, item, timeoutMs) -> int

Puts a text item in the queue; if the queue is full, it WAITS until room frees up or the time limit expires.

Parameters
Parameter Type Description
name string The name of the queue.
item string The text to insert.
timeoutMs int Maximum wait in milliseconds; 0 or negative = infinite.
Return value

1 if it was inserted; 0 if the time limit expired without room freeing up.

Example
queue.Create("q", 3);
queue.Push("q", "a");
queue.Push("q", "b");
queue.Push("q", "c");
queue.Pop("q"); // one slot frees up
printf("%d\n", queue.WaitPush("q", "e", 100));
queue.Destroy("q");
Output after running
1
When to use

For backpressure: if the producer is faster than the consumer, WaitPush slows it down and keeps the queue from growing without bound. The time limit avoids getting stuck forever.

queue.PushBlob

queue.PushBlob(name, ref item) -> int

Puts a binary (blob) item at the end of the queue. Non-blocking.

Parameters
Parameter Type Description
name string The name of the queue.
item ref blob The blob to insert, passed by reference (the $-prefixed variable directly).
Return value

1 if it was inserted; 0 if the queue was full.

Example
blob $B[8];
queue.Create("q", 3);
$B[0] = 7; $B[1] = 8; $B[2] = 9;
$B.Length = 3;
printf("%d\n", queue.PushBlob("q", $B));
queue.Destroy("q");
Output after running
1
When to use

For passing raw bytes (a network packet, serialized data) between threads. The blob must be passed by reference; the plugin copies it on insertion.

queue.WaitPushBlob

queue.WaitPushBlob(name, ref item, timeoutMs) -> int

Puts a binary item in the queue; on a full queue it WAITS with the given time limit.

Parameters
Parameter Type Description
name string The name of the queue.
item ref blob The blob to insert, by reference.
timeoutMs int Maximum wait in milliseconds; 0 or negative = infinite.
Return value

1 if it was inserted; 0 if the time limit expired.

Example
blob $B[8];
queue.Create("q", 3);
$B[0] = 7; $B.Length = 1;
printf("%d\n", queue.WaitPushBlob("q", $B, 100));
queue.Destroy("q");
Output after running
1
When to use

The blob counterpart of WaitPush: for passing binary data with backpressure, with a time limit to avoid getting stuck.

Taking out (consumer side)

Taking an item from the front of the queue. Pop is non-blocking (an empty queue gives an empty value); WaitPop sleeps until an item arrives.

queue.Pop

queue.Pop(name) -> string

Takes the oldest text item from the front of the queue. Non-blocking: on an empty queue it returns an empty string at once.

Parameters
Parameter Type Description
name string The name of the queue.
Return value

The taken text; an empty string if the queue is empty. If the item at the front is not a string (but a blob), a runtime error — use PopBlob.

Example
queue.Create("q", 3);
queue.Push("q", "a");
queue.Push("q", "b");
// FIFO: a comes out first
printf("%s\n", queue.Pop("q"));
printf("%s\n", queue.Pop("q"));
printf("[%s]\n", queue.Pop("q"));
queue.Destroy("q");
Output after running
a
b
[]
When to use

Good when you do not want to wait: in a loop you check whether an item is there, and if you get an empty string you do something else. It is hard to tell apart from a real empty item, so avoid putting empty strings in the queue.

queue.WaitPop

queue.WaitPop(name, timeoutMs) -> string

Takes the oldest text item; if the queue is empty, it SLEEPS and waits until an item arrives or the time limit expires.

Parameters
Parameter Type Description
name string The name of the queue.
timeoutMs int Maximum wait in milliseconds; 0 or negative = infinite.
Return value

The taken text; an empty string if the time limit expired without an item arriving.

Example
queue.Create("q", 3);
queue.Push("q", "c");
printf("%s\n", queue.WaitPop("q", 100));
// empty queue: timeout after 50 ms
printf("[%s]\n", queue.WaitPop("q", 50));
queue.Destroy("q");
Output after running
c
[]
When to use

The main tool of a consumer thread: it waits at 0% CPU and wakes immediately when a producer inserts an item. With a time limit you can periodically check a stop signal so the thread can exit cleanly.

queue.PopBlob

queue.PopBlob(name) -> blob

Takes the oldest binary item from the front of the queue. Non-blocking.

Parameters
Parameter Type Description
name string The name of the queue.
Return value

The taken blob; a 0-length blob if the queue is empty.

Example
blob $B[8];
blob $Out[16];
queue.Create("q", 3);
$B[0] = 7; $B[1] = 8; $B[2] = 9; $B.Length = 3;
queue.PushBlob("q", $B);
$Out = queue.PopBlob("q");
printf("%d\n", $Out.Length);
printf("%d\n", $Out[0]);
queue.Destroy("q");
Output after running
3
7
When to use

For taking out binary items. Tell an empty result apart from a 0-length item by the length (.Length); an empty queue always gives length 0.

queue.WaitPopBlob

queue.WaitPopBlob(name, timeoutMs) -> blob

Takes the oldest binary item; on an empty queue it WAITS with the given time limit.

Parameters
Parameter Type Description
name string The name of the queue.
timeoutMs int Maximum wait in milliseconds; 0 or negative = infinite.
Return value

The taken blob; a 0-length blob if the time limit expired.

Example
blob $B[8];
blob $Out[16];
queue.Create("q", 3);
$B[0] = 7; $B.Length = 1;
queue.PushBlob("q", $B);
$Out = queue.WaitPopBlob("q", 100);
printf("%d\n", $Out.Length);
queue.Destroy("q");
Output after running
1
When to use

The blob counterpart of WaitPop: for a consumer thread waiting on binary items, with efficient, spin-free waiting.

Inspection

Querying the current state of the queue.

queue.Size

queue.Size(name) -> int

Returns the number of items currently waiting in the queue.

Parameters
Parameter Type Description
name string The name of the queue.
Return value

The current number of items. A non-existent queue causes a runtime error.

Example
queue.Create("q", 3);
queue.Push("q", "a");
queue.Push("q", "b");
printf("%d\n", queue.Size("q"));
queue.Destroy("q");
Output after running
2
When to use

For monitoring the queue's load (for logging or a backpressure decision). With multiple threads the size is a snapshot — by the time you use it, it may have changed.

queue.IsEmpty

queue.IsEmpty(name) -> int

Tells whether the queue is empty.

Parameters
Parameter Type Description
name string The name of the queue.
Return value

1 if the queue is empty; 0 if it has items.

Example
queue.Create("q", 3);
printf("%d\n", queue.IsEmpty("q"));
queue.Push("q", "a");
printf("%d\n", queue.IsEmpty("q"));
queue.Destroy("q");
Output after running
1
0
When to use

More readable than a Size == 0 test. Useful as a loop condition in single-threaded processing; with multiple threads, rely on WaitPop's time limit instead.

queue.Capacity

queue.Capacity(name) -> int

Returns the queue's maximum item count (as you gave it to Create).

Parameters
Parameter Type Description
name string The name of the queue.
Return value

The maximum item count; 0 if the queue is unbounded.

Example
queue.Create("u", 0);
// 0 = unbounded
printf("%d\n", queue.Capacity("u"));
queue.Destroy("u");
Output after running
0
When to use

For querying the queue's configured limit. A 0 return signals an unbounded queue, where Push never returns full (0).

Practical notes

What the queue plugin is good for

  • Producer-consumer tasks: one or more threads put in work, others process it.

  • Message passing and work distribution between threads, with safe handoff.

  • Backpressure: with a bounded queue you can slow down a producer that is too fast.

  • An event queue on which the processing thread waits efficiently (0% CPU) for the next item.

Blocking vs non-blocking pattern

In a background thread, almost always use the Wait variants: WaitPop waits while asleep and wakes immediately on a new item, so it does not needlessly spin the CPU. The non-blocking Push/Pop are good when the caller does not want to wait — for example a single-threaded processor that polls several queues in turn. By giving the Wait calls a time limit, the thread can periodically check a stop signal and exit cleanly.

Common pitfalls

  • Non-existent queue: Push/Pop/Size/Capacity raise a runtime error for it (only Exists/Destroy tolerate it). Always create the queue before using it.

  • Kind mismatch: a string Pop fails on a blob item (and vice versa). Keep a single item kind per queue.

  • Empty string as an item: Pop signals an empty queue with an empty string too, so the two are hard to tell apart — avoid putting empty strings in the queue, or use Size/IsEmpty.

  • Destroy and waiters: Destroy wakes but does not wait for pending Wait calls. Stop the worker threads first, only then call Destroy.

Error handling

A wrong argument count or type, a non-existent queue (for Push/Pop/Size/Capacity), an already-taken or invalid name (Create), and out of memory are reported by the runtime as a runtime error, and the script stops. The usual content-level cases, however, are signaled by a neutral return value: a full queue gives 0 on Push, an empty queue gives an empty string or a 0-length blob on Pop, and a timeout gives the same 0/empty. So the usual cases need no extra guard checks.

The log logging plugin

Structured logging with levels, sinks, and rotation