🦋 108. Basic usage of NativeCall in Perl 6

NativeCall is both a module and a technology in Perl 6 that allows you to call C functions from your Perl 6 code. Today, let’s meet the most basic usage.

Take the rand() function from the C standard library:

#include <stdio.h>
#include <stdlib.h>

int main() {    
    int r = rand();
    printf("%i\n", r);

    return 0;
}

If you are used to Perl, it may be a surprise that the program actually prints the same number every time you call it. (That’s a kind of surprise of forgotten knowledge.) Compile and run:

$ gcc rand.c

$ ./a.out 
16807

$ ./a.out 
16807

Now let’s call C’s rand() from Perl 6. Refer to the documentation of NativeCall to see the options it offers.

Your Perl 6 program may look like this:

use NativeCall;

sub c_rand() returns int32 is native('c') is symbol('rand') {*}

say c_rand();

Although the rand() function was chosen as one from the standard library that do not need parameters, the choice added a few small complications to the Perl 6 code. But it is good that we can demonstrate them all together.

Similarly to how you declare external functions in C, you need to tell Perl 6 that there is some function that it should load from an external library. The name of the library is passed in the native trait. Its argument, c, will be converted to libc, and the corresponding shared library will be searched for in the standard places on your computer.

The symbol trait tells us the original function name. It is a bit weird to me, but Rakudo cannot handle the following code where you use the same name as in C:

use NativeCall;

sub rand() returns int32 is native('c') {*}

say rand();

This is how you would declare an external C function if you didn’t want to rename it in your program. Unfortunately, Perl 6 also has rand, and we got an error:

===SORRY!=== Error while compiling rand.pl6
 Unsupported use of rand(); in Perl 6 please use rand
 at rand.pl6:5
 ------> say rand⏏();

Finally, returns int32 tells that the function returns a native 32-bit integer.

Another surprise pops up when you run our Perl 6 program. It returns a random value, and that value is different each time you call it:

$ perl6 rand.pl6 
1790432239

$ perl6 rand.pl6 
1431059869

Is it really working? It is random, but why it does not copy the behaviour of the reference C program?

Does Rakudo call srand when it starts up? A shallow investigation gives the following log comment:

nqp/MoarVM/docs/ChangeLog:+ Do not call srand() if not using rand()

Let’s go back to our test case. To make sure it works, let us create our own version of the function, which will be returning the same value again and again:

myrand.c:

int myrand() {
    return 42;
}

myrand.h:

int myrand();

Test it in pure C first:

#include "myrand.h"
#include <stdio.h>

int main() {
    int r = myrand();
    printf("%i\n", r);

    return 0;
}
$ gcc myrand-main.c myrand.c 

$ ./a.out 
42

$ ./a.out 
42

It works. Now use it from Perl 6, and the program is almost the same as before:

use NativeCall;

sub myrand() returns int32 is native('libmyrand.so') {*}

say myrand();

The main differences are the name of the function (there is no clash with Perl 6 built-in functions, and thus no need to introduce an alias) and the library name, which is a file name this time.

(This is how you create a shared library:)

$ gcc -shared -olibmyrand.so myrand.c 

If you run the updated Perl 6 program, you will only see 42 in the output.

$ perl6 myrand.pl6 
42

$ perl6 myrand.pl6 
42

This confirms that the returns int32 clause works correctly, and the value returned from the C library is understood by Perl 6.

🦋 74. Typed hashes in Perl 6

In Perl 6, you can restrict the content of a variable container by specifying its type, for example:

my Int $i;

There is only one value in a scalar variable. You can extend the concept to arrays and let its element to keep only integers, as it is done in the next example:

> my Int @i;
[]

> @i.push(42);
[42]

> @i.push('Hello');
Type check failed in assignment to @i;
expected Int but got Str ("Hello")
  in block <unit> at <unknown file> line 1

Hashes keeps pairs, so you can specify the type of both keys and values. The syntax is not deductible from the above examples.

First, let us announce the type of the value:

my Str %s;

Now, it is possible to have strings as values:

> %s<Hello> = 'World'
World

> %s<42> = 'Fourty-two'
Fourty-two

But it’s not possible to save integers:

> %s<x> = 100
Type check failed in assignment to %s;
expected Str but got Int (100)
  in block <unit> at <unknown file> line 1

(By the way, notice that in the case of %s<42> the key is a string.)

To specify the type of the second dimension, namely, of the hash keys, give the type in curly braces:

my %r{Rat};

This variable is also referred to as object hash.

Having this, Perl expects you to have Rat keys for this variable:

> %r<22/7> = pi
3.14159265358979

> %r
{22/7 => 3.14159265358979}

Attempts to use integers or strings, for example, fail:

> %r<Hello> = 1
Type check failed in binding to parameter 'key';
expected Rat but got Str ("Hello")
  in block <unit> at <unknown file> line 1

> %r{23} = 32
Type check failed in binding to parameter 'key';
expected Rat but got Int (23)
  in block <unit> at <unknown file> line 1

Finally, you can specify the types of both keys and values:

my Str %m{Int};

This variable can be used for translating month number to month names but not vice versa:

> %m{3} = 'March'
March

> %m<March> = 3
Type check failed in binding to parameter 'key';
expected Int but got Str ("March")
  in block <unit> at <unknown file> line 1

 

🦋 69. Setting timeouts in Perl 6

In Perl 5, I used to set timeouts using signals (or, at least, that was an easy and predictable way). In Perl 6, you can use promises. Let us see how to do that.

To imitate a long-running task, create an infinite loop that prints its state now and then. Here it is:

for 1 .. * {
    .say if $_ %% 100_000;
}

As soon as the loop gets control, it will never quit. Our task is to stop the program in a couple of seconds, so the timer should be set before the loop:

Promise.in(2).then({
    exit;
});

for 1 .. * {
    .say if $_ %% 100_000;
}

Here, the Promise.in method creates a promise that is automatically kept after the given number of seconds. On top of that promise, using then, we add another promise, whose code will be run after the timeout. The only statement in the body here is exit that stops the main program.

Run the program to see how it works:

$ time perl6 timeout.pl
100000
200000
300000
. . .
3700000
3800000
3900000

real 0m2.196s
user 0m2.120s
sys 0m0.068s

The program counts up to about four millions on my computer and quits in two seconds. That is exactly the behaviour we needed.

For comparison, here is the program in Perl 5:

use v5.10;

alarm 2;
$SIG{ALRM} = sub {
    exit;
};

for (my $c = 1; ; $c++) {
    say $c unless $c % 1_000_000;
}

(It manages to count up to 40 million, but that’s another story.)

🦋 66. Atomic operations in Perl 6

N. B. The examples below require a fresh Rakudo compiler, at least of the version 2017.09.

Discussing parallel computing earlier or later leads to solving race conditions. Let us look at a simple counter that is incremented by two parallel threads:

my $c = 0;

await do for 1..10 {
    start {
        $c++ for 1 .. 1_000_000
    }
}

say $c;

If you run the program a few times, you will immediately see that the results are very different:

$ perl6 atomic-1.pl 
3141187
$ perl6 atomic-1.pl 
3211980
$ perl6 atomic-1.pl 
3174944
$ perl6 atomic-1.pl 
3271573

Of course, the idea was to increase the counter by 1 million in all of the ten threads, but about ⅓ of the steps were lost. It is quite easy to understand why that happens: the parallel threads read the variable and write to it ignoring the presence of other threads and not thinking that the value can be changed in-between. Thus, some of the threads work with an outdated value of the counter.

Perl 6 offers a solution: atomic operations. The syntax of the language is equipped with the Atom Symbol (U+0x269B) ⚛ character (no idea of why it is displayed in that purple colour). Instead of $c++, you should type $c⚛++.

my atomicint $c = 0;

await do for 1..10 {
    start {
        $c⚛++ for 1 .. 1_000_000
    }
}

say $c;

And before thinking of the necessity to use a Unicode character, let us look at the result of the updated program:

$ perl6 atomic-2.pl 
10000000

This is exactly the result we wanted!

Notice also, that the variable is declared as a variable of the atomicint type. That is a synonym for int, which is a native integer (unlike Int, which is a data type represented by a Perl 6 class).

It is not possible to ask a regular value to be atomic. That attempt will be rejected by the compiler:

$ perl6 -e'my $c; $c⚛++'
Expected a modifiable native int argument for '$target'
  in block  at -e line 1

A few other operators can be atomic, for example, prefix and postfix increments and decrements ++ and --, or += and -=. There are also atomic versions of the assignment operator = and the one for reading: ⚛ (sic!).

If you need atomic operations in your code, you are not forced to use the ⚛ character. There exists a bunch of alternative functions that you can use instead of the operators:

my atomicint $c = 1;

my $x = ⚛$c;  $x = atomic-fetch($c);
$c ⚛= $x;     atomic-assign($c, $x);
$c⚛++;        atomic-fetch-inc($c);
$c⚛--;        atomic-fetch-dec($c);
++⚛$c;        atomic-inc-fetch($c);
--⚛$c;        atomic-dec-fetch($c);
$c ⚛+= $x;    atomic-fetch-add($c,$x);

say $x; # 1
say $c; # 3

🦋64. What does gist do in Perl 6?

When you print an object, say, as say $x, Perl 6 calls the gist method. This method is defined for all built-in types: for some of them, it calls the Str method, for some the perl method, for some types it makes the string representation somehow differently.

Let us see how you can use the method to create your own variant:

class X {
    has $.value;

    method gist {
        '[' ~ $!value ~ ']'
    }
}

my $x = X.new(value => 42);

say $x; # [42]
$x.say; # [42]

When you call say, the program prints a number in square brackets: [42].

Please notice that the interpolation inside double-quoted strings is using Str, not gist. You can see it here:

say $x.Str; # X<140586830040512>
say "$x";   # X<140586830040512>

If you need a custom interpolation, redefine the Str method:

class X {
    has $.value;

    method gist {
        '[' ~ $!value ~ ']'
    }
    method Str {
        '"' ~ $!value ~ '"'
    }
}

my $x = X.new(value => 42);

say $x;     # [42]
$x.say;     # [42]

say $x.Str; # "42"
say "$x";   # "42"

Now, we got the desired behaviour.

🦋63. More on the proto keyword in Perl 6

Before digging into the details of the EVAL routine, we have to reveal some more information about protos and multiple dispatch. Examine the following program:

proto sub f($x) {
    say "proto f($x)";
}

multi sub f($x) {
    say "f($x)"
}

multi sub f(Int $x) {
    say "f(Int $x)"
}

multi sub f(Str $x) {
    say "f(Str $x)"
}

f(2);
f('2');
f(3);
f('3');

Here, there are three multi-candidates of the function plus a function declared with the proto keyword. Earlier, we only saw such proto-functions with empty body, such as:

proto sub f($x) {*}

But this is not a necessity. The function can carry a regular load, as we see in the example:

proto sub f($x) {
    say "proto f($x)";
}

Run the program:

proto f(2)
proto f(2)
proto f(3)
proto f(3)

All the calls were caught by the proto-candidate. Now, update it and return the {*} block for some dedicated values;

proto sub f($x) {
    if $x.Str eq '3' {
        return {*}
    }
    say "proto f($x)";
}

The if check triggers its block for the last two function calls:

f(3);
f('3');

In these cases, the proto-function returns {*}, which makes Perl 6 trying other candidates. As we have enough candidates for both integer and string arguments, the compiler can easily choose one of them:

proto f(2)
proto f(2)
f(Int 3)
f(Str 3)

🦋62. The EVAL routine in Perl 6, part 1

The EVAL routine in Perl 6 compiles and executes the code that it gets as an argument.  Today, we will see some potential use cases that you may try in your practice. Tomorrow, we will dig into Rakudo sources to see how it works and why it breaks sometimes.

1

Let us start with evaluating a simple program:

EVAL('say 123');

This program prints 123, there’s no surprise here.

2

There are, though, more complicated cases. What, do you think, does the following program print?

EVAL('say {456}');

I guess it prints not what you expected:

-> ;; $_? is raw { #`(Block|140570649867712) ... }

It parses the content between the curly braces as a pointy block.

3

What if you try double quotes?

EVAL("say {789}");

Now it even refuses to compile:

===SORRY!=== Error while compiling eval.pl
EVAL is a very dangerous function!!! (use the MONKEY-SEE-NO-EVAL pragma to override this error,
but only if you're VERY sure your data contains no injection attacks)
at eval.pl:6
------> EVAL("say {789}")⏏;

4

We can fix the code by adding a few magic words:

use MONKEY-SEE-NO-EVAL;

EVAL("say {789}");

This time, it prints 789.

5

The code is executed (we don’t know yet when exactly, that is the topic of tomorrow’s post), so you can make some calculations, for example:

use MONKEY-SEE-NO-EVAL;

EVAL("say {7 / 8 + 9}"); # 9.875

6

Finally, if you try passing a code block directly, you also cannot achieve the goal, even with a blind monkey:

use MONKEY-SEE-NO-EVAL;

EVAL {say 123};

The error happens at runtime:

Constraint type check failed in binding to parameter '$code';
expected anonymous constraint to be met but got 
-> ;; $_? is raw { #`...
  in block <unit> at eval.pl line 10

This message looks cryptic, but at least we see once again that we got an anonymous pointy block passed to the function.

7

And before we wrap up for today, an attempt to use Perl 5 syntax:

eval('say 42');

There is no such function in Perl 6, and we get a standard error message:

===SORRY!=== Error while compiling eval2.pl
Undeclared routine:
  eval used at line 5. Did you mean 'EVAL', 'val'?

It looks OK but it can be better.

Stay tuned, tomorrow we will try to understand how all these examples work in Rakudo.