63. More on the proto keyword in Perl 6

Before digging into the details of the EVAL routine, we have to reveal some more information about protos and multiple dispatch. Examine the following program:

proto sub f($x) {
    say "proto f($x)";
}

multi sub f($x) {
    say "f($x)"
}

multi sub f(Int $x) {
    say "f(Int $x)"
}

multi sub f(Str $x) {
    say "f(Str $x)"
}

f(2);
f('2');
f(3);
f('3');

Here, there are three multi-candidates of the function plus a function declared with the proto keyword. Earlier, we only saw such proto-functions with empty body, such as:

proto sub f($x) {*}

But this is not a necessity. The function can carry a regular load, as we see in the example:

proto sub f($x) {
    say "proto f($x)";
}

Run the program:

proto f(2)
proto f(2)
proto f(3)
proto f(3)

All the calls were caught by the proto-candidate. Now, update it and return the {*} block for some dedicated values;

proto sub f($x) {
    if $x.Str eq '3' {
        return {*}
    }
    say "proto f($x)";
}

The if check triggers its block for the last two function calls:

f(3);
f('3');

In these cases, the proto-function returns {*}, which makes Perl 6 trying other candidates. As we have enough candidates for both integer and string arguments, the compiler can easily choose one of them:

proto f(2)
proto f(2)
f(Int 3)
f(Str 3)

62. The EVAL routine in Perl 6, part 1

The EVAL routine in Perl 6 compiles and executes the code that it gets as an argument.  Today, we will see some potential use cases that you may try in your practice. Tomorrow, we will dig into Rakudo sources to see how it works and why it breaks sometimes.

1

Let us start with evaluating a simple program:

EVAL('say 123');

This program prints 123, there’s no surprise here.

2

There are, though, more complicated cases. What, do you think, does the following program print?

EVAL('say {456}');

I guess it prints not what you expected:

-> ;; $_? is raw { #`(Block|140570649867712) ... }

It parses the content between the curly braces as a pointy block.

3

What if you try double quotes?

EVAL("say {789}");

Now it even refuses to compile:

===SORRY!=== Error while compiling eval.pl
EVAL is a very dangerous function!!! (use the MONKEY-SEE-NO-EVAL pragma to override this error,
but only if you're VERY sure your data contains no injection attacks)
at eval.pl:6
------> EVAL("say {789}")⏏;

4

We can fix the code by adding a few magic words:

use MONKEY-SEE-NO-EVAL;

EVAL("say {789}");

This time, it prints 789.

5

The code is executed (we don’t know yet when exactly, that is the topic of tomorrow’s post), so you can make some calculations, for example:

use MONKEY-SEE-NO-EVAL;

EVAL("say {7 / 8 + 9}"); # 9.875

6

Finally, if you try passing a code block directly, you also cannot achieve the goal, even with a blind monkey:

use MONKEY-SEE-NO-EVAL;

EVAL {say 123};

The error happens at runtime:

Constraint type check failed in binding to parameter '$code';
expected anonymous constraint to be met but got 
-> ;; $_? is raw { #`...
  in block <unit> at eval.pl line 10

This message looks cryptic, but at least we see once again that we got an anonymous pointy block passed to the function.

7

And before we wrap up for today, an attempt to use Perl 5 syntax:

eval('say 42');

There is no such function in Perl 6, and we get a standard error message:

===SORRY!=== Error while compiling eval2.pl
Undeclared routine:
  eval used at line 5. Did you mean 'EVAL', 'val'?

It looks OK but it can be better.

Stay tuned, tomorrow we will try to understand how all these examples work in Rakudo.

61. Declared in BOOTSTRAP

First of all, a new release of the Rakudo Perl 6 compiler was announced today: 2018.02. There are many fixes and speed improvements there, including one proposed by me. Let me not go through the changes, as most of them require quite in-depth knowledge of the Rakudo internals.

Instead, let us take a low-hanging fruit and look at the feature that you may see almost immediately when you start reading Rakudo sources.

Ideologically, Perl 6 can (and should) be written in Perl 6. Currently, some parts are written in NQP but still, the vast number of data types—located in the src/core directory—are implemented in Perl 6.

The thing is that some classes are not fully defined there. Or their relation to other classes is not explicit. For example, here’s the whole definition of the Sub class:

my class Sub { # declared in BOOTSTRAP
    # class Sub is Routine

}

Not only you don’t see any methods here, but also its hierarchy is defined ‘via comments.’ Of course, Perl 6 is not that smart to read comments saying ‘make this code great and cool,’ so let’s see what’s going on here.

In the source tree, there is the following file: src/Perl6/Metamodel/BOOTSTRAP.nqp, where the above-mentioned relation is built.

The class itself (the Sub name) is declared as a so-called stub in the very beginning of the file:

my stub Sub metaclass Perl6::Metamodel::ClassHOW { ... };

Now, the name is known but the definition is not yet ready. We have seen a few examples earlier. Here is the part of the Sub class:

# class Sub is Routine {
Sub.HOW.add_parent(Sub, Routine);
Sub.HOW.compose_repr(Sub);
Sub.HOW.compose_invocation(Sub);

This code lets the user think that the class definition is the following, as the documentation says:

class Sub is Routine {
}

Other examples of Routine children are Method, Submethod, and Macro. The first two are also defined in BOOTSTRAP:

# class Method is Routine {
Method.HOW.add_parent(Method, Routine);
Method.HOW.compose_repr(Method);
Method.HOW.compose_invocation(Method);

# class Submethod is Routine {
Submethod.HOW.add_parent(Submethod, Routine);
Submethod.HOW.compose_repr(Submethod);
Submethod.HOW.compose_invocation(Submethod);

The classes themselves are defined in their corresponding files src/core/Method.pm and src/core/Submethod.pm:

my class Method { # declared in BOOTSTRAP
    # class Method is Routine

    multi method gist(Method:D:) { self.name }
}
my class Submethod { # declared in BOOTSTRAP
    # class Submethod is Routine

    multi method gist(Submethod:D:) { self.name }
}

Unlike them, the Marco type’s hierarchy is explicitly announced in src/core/Macro.pm:

my class Macro is Routine {
}

As you may see, the classes basically introduce their namespaces and do not add many methods to their Routine parent.

The Routine class in its turn is also defined in two places: in src/core/Routine.pm and in BOOTSTRAP.pm.

my class Routine { # declared in BOOTSTRAP
    # class Routine is Block
    # has @!dispatchees;
    # has Mu $!dispatcher_cache;
    # has Mu $!dispatcher;
    # has int $!rw;
    # has Mu $!inline_info;
    # has int $!yada;
    # has Mu $!package;
    # has int $!onlystar;
    # has @!dispatch_order;
    # has Mu $!dispatch_cache;

This time, there are many methods, some of which are added in src/core/Routine.pm using regular Perl 6 syntax, and some are added through BOOTSTRAP in NQP:

In Perl 6:

method candidates() {
    self.is_dispatcher ??
        nqp::hllize(@!dispatchees) !!
        (self,)
}

In NQP:

Routine.HOW.add_method(Routine, 'dispatcher', nqp::getstaticcode(sub ($self) {
    nqp::getattr(nqp::decont($self),
        Routine, '$!dispatcher')
    }));

Similarly, the attributes from comments are created in NQP:

Routine.HOW.add_attribute(Routine, Attribute.new(:name<@!dispatchees>, :type(List), :package(Routine)));
Routine.HOW.add_attribute(Routine, Attribute.new(:name<$!dispatcher_cache>, :type(Mu), :package(Routine)));

As far as I understand, such bootstrapping is needed because Rakudo requires some Perl 6 defined before it can compile itself. For example, if you declare Sub’s relation to Routine completely in src/core/Sub.pm, then you get an error when compiling Rakudo:

Representation for Sub must be composed before it can be serialized

60. Examining the Real role of Perl 6, part 3

As promised yesterday, let us take a look at the two methods of the Real role: polymod and base.

polymod

I already devoted a post to the Int.polymod method, but the method also exists in the Real role. Let us see if it is different.

method polymod(Real:D: +@mods) {
    my $more = self;
    my $lazy = @mods.is-lazy;
    fail X::OutOfRange.new(
        :what('invocant to polymod'), :got($more), :range<0..Inf>
    ) if $more < 0; 
    gather { 
        for @mods -> $mod {
            last if $lazy and not $more;
            Failure.new(X::Numeric::DivideByZero.new:
                using => 'polymod', numerator => $more
            ) unless $mod;
            take my $rem = $more % $mod;
            $more -= $rem;
            $more /= $mod;
        }
        take $more if ($lazy and $more) or not $lazy;
    }
}

It looks familiar. Comparing to the method of Int, the separation of lazy and non-lazy lists is incorporated in the main loop. In the rest, it is again the mod operation (in the form of %) and a division (and some additional subtraction).

Try the method on the same 120 (but as a Numeric value):

> say 120.polymod(10,10)
(0 2 1)

> say 120e0.polymod(10,10)
(0 2 1)

The first method is a call of Int.polymod, while the second one is Real.polymod. The results are the same.

A final note on the method. Just notice that it also works with non-integer values:

> 120.34.polymod(3.3, 4.4)
(1.54 0.8 8)

Indeed, 1.54 + 0.8 * 3.3 + 8 * 3.3 * 4.4 = 120.34.

base

The base method converts a number to its representation in a different system, e. g., hexadecimal, octal, or in a system with 5 or 35 digits. Extrapolating hexadecimal system, you may guess that if there are 36 digits, then the digits are 0 to 9 and A to Z.

A few examples with the numbers with a floating point (actually, Rat numbers here):

> 120.34.base(10)
120.34
> 120.34.base(36)
3C.C8N1FU
> 120.34.base(3)
11110.100012
> 120.34.base(5)
440.132223

The fractional part is converted separately. The second argument of the method limits the number of digits in it. Compare:

> 120.34.base(5)
440.132223
> 120.34.base(5, 2)
440.14

I will skip the details of the method internals and will only show the most interesting parts.

The signature of the method in the src/core/Real.pm file is the following:

 method base(Int:D $base, $digits? is copy)

The documentation interprets that quite differently (although correct semantically):

method base(Real:D: Int:D $base where 2..36, $digits? --> Str:D)

The possible digits are listed explicitly (not in ranges):

my @conversion := <0 1 2 3 4 5 6 7 8 9
                   A B C D E F G H I J
                   K L M N O P Q R S T
                   U V W X Y Z>;

Finally, the last gathering of the separate digits into a resulting string is done like that, using a call to the Int.base method:

my Str $r = $int_part.base($base);
$r ~= '.' ~ @conversion[@frac_digits].join if @frac_digits;
# if $int_part is 0, $int_part.base doesn't see the sign of self
$int_part == 0 && self < 0 ?? '-' ~ $r !! $r;

The method also does some heuristics to determine the number of digits after the floating point:

my $prec = $digits // 1e8.log($base.Num).Int;
. . .
for ^$prec {
    last unless $digits // $frac;
    $frac = $frac * $base;
    push @frac_digits, $frac.Int;
    $frac = $frac - $frac.Int;
}

Compare now the method with the same method from the Int class:

multi method base(Int:D: Int:D $base) {
    2 <= $base <= 36
        ?? nqp::p6box_s(nqp::base_I(self,nqp::unbox_i($base)))
        !! Failure.new(X::OutOfRange.new(
            what => "base argument to base", :got($base), :range<2..36>))
}

In this case, all the hard work is delegated to the base_I function of NQP.

And that’s more or less all that I wanted to cover from the Real role internals.

59. Examining the Real role of Perl 6, part 2

Today, we continue our initial exploration of the Real role, that was started a couple of days ago.

Together with its methods, the role contains a number of subroutines (placed outside the role) that define the infix operators with the objects of the Real type. The list is not that long, so let me copy it here:

multi sub infix:<+>(Real \a, Real \b) { a.Bridge + b.Bridge }
multi sub infix:<->(Real \a, Real \b) { a.Bridge - b.Bridge }
multi sub infix:<*>(Real \a, Real \b) { a.Bridge * b.Bridge }
multi sub infix:</>(Real \a, Real \b) { a.Bridge / b.Bridge }
multi sub infix:<%>(Real \a, Real \b) { a.Bridge % b.Bridge }
multi sub infix:<**>(Real \a, Real \b) { a.Bridge ** b.Bridge }
multi sub infix:«<=>»(Real \a, Real \b) { a.Bridge <=> b.Bridge }
multi sub infix:<==>(Real \a, Real \b) { a.Bridge == b.Bridge }
multi sub infix:«<»(Real \a, Real \b) { a.Bridge < b.Bridge }
multi sub infix:«<=»(Real \a, Real \b) { a.Bridge <= b.Bridge }
multi sub infix:«≤» (Real \a, Real \b) { a.Bridge ≤ b.Bridge }
multi sub infix:«>»(Real \a, Real \b) { a.Bridge > b.Bridge }
multi sub infix:«>=»(Real \a, Real \b) { a.Bridge >= b.Bridge }
multi sub infix:«≥» (Real \a, Real \b) { a.Bridge ≥ b.Bridge }

proto sub infix:<mod>($, $) is pure {*}
multi sub infix:<mod>(Real $a, Real $b) {
    $a - ($a div $b) * $b;
}

As you see, most of the operators are using the Bridge method, which allows using the same code in derived classes that may redefine the bridge.

There’s also one prefix operation for negation:

multi sub prefix:<->(Real:D \a) { -a.Bridge }

The cis function works as a type converter returning a complex number:

roto sub cis($) {*}
multi sub cis(Real $a) { $a.cis }

Try it out:

> cis(pi)
-1+1.22464679914735e-16i

> cis(pi).WHAT
(Complex)

A bit outstanding, there is a function for atan2:

proto sub atan2($, $?) {*}
multi sub atan2(Real \a, Real \b = 1e0) { a.Bridge.atan2(b.Bridge) }
# should really be (Cool, Cool), and then (Cool, Real) and (Real, Cool)
# candidates, but since Int both conforms to Cool and Real, we'd get lots
# of ambiguous dispatches. So just go with (Any, Any) for now.
multi sub atan2( \a, \b = 1e0) { a.Numeric.atan2(b.Numeric) }

It is a bit strange as it does not follow the manner in which other trigonometric functions are implemented. The atan2 routine is also defined as a method:

proto method atan2(|) {*}
multi method atan2(Real $x = 1e0) { self.Bridge.atan2($x.Bridge) }
multi method atan2(Cool $x = 1e0) { self.Bridge.atan2($x.Numeric.Bridge) }

All other trigonometric functions only exist as Real methods. The rest is defined inside the src/core/Numeric.pm file as self-standing subroutines, for example:

proto sub cos($) is pure {*}
multi sub cos(Numeric \x) { x.cos }
multi sub cos(Cool \x) { x.Numeric.cos }

. . .

proto sub atan($) is pure {*}
multi sub atan(Numeric \x) { x.atan }
multi sub atan(Cool \x) { x.Numeric.atan }

There are a few more routines but let us skip them, as they are quite straightforward and clear.

In the next part, we will explore the two methods of the Real role: polymod and base. Stay tuned!

58. A word on polymod in Perl 6

Before moving to the second part of the Real role, let us stop on the polymod method of the Int class.

The method takes a number and a list of arbitrary numbers (units) and returns the corresponding multipliers. So that you can easily say that 550 seconds, for example, is 9 minutes and 10 seconds:

> 550.polymod(60)
(10 9)

In the method call, the value of 60 is the number of seconds in a minute. In the result, 9 is a number of minutes, and 10 is a remainder, which is a number of seconds. So, 550 seconds = 10 second + 9 minutes.

If you want more details, add more units. For example, what is it 32768 seconds?

> 32768.polymod(60, 60, 24)
(8 6 9 0)

It is 8 seconds, 6 minutes, 9 hours, and 0 days.

Similarly, 132768 seconds are 1 day, 12 hours, 52 minutes, and 48 seconds:

> 132768.polymod(60, 60, 24)
(48 52 12 1)

Honestly, it was quite difficult for me to understand how it works and how to read the result.

Another example from the documentation was even harder:

> 120.polymod(1, 10, 100)
(0 0 12 0)

What does 12 mean? It is, obviously, 12 times 10. OK, But I asked to give me some information about the number of hundreds. My expectation is to have it like that: 120 is 2 times 10 and 1 time 100.

Try 121:

> 121.polymod(1, 10)
(0 1 12)

Erm, why zero? Zero plus 1 times 1 plus 12 times 10? Brr. Ah! You don’t need to specify an explicit 1 in the arguments:

> 121.polymod(10)
(1 12)

That makes more sense. Except the fact that I still don’t know how many hundreds are there in 121:

> 121.polymod(10, 100)
(1 12 0)
> 121.polymod(100, 10)
(21 1 0)

It’s time to take a look at the source code (src/core/Int.pm):

method polymod(Int:D: +@mods) {
    fail X::OutOfRange.new(
        :what('invocant to polymod'), :got(self), :range<0..^Inf>
    ) if self < 0; 

    gather { 
         my $more = self; 
         if @mods.is-lazy { 
             for @mods -> $mod {
                $more
                    ?? $mod
                    ?? take $more mod $mod
                    !! Failure.new(X::Numeric::DivideByZero.new:
                            using => 'polymod', numerator => $more)
                    !! last;
                $more = $more div $mod;
            }
            take $more if $more;
        }
        else {
            for @mods -> $mod {
                $mod
                    ?? take $more mod $mod
                    !! Failure.new(X::Numeric::DivideByZero.new:
                        using => 'polymod', numerator => $more);
                $more = $more div $mod;
            }
            take $more;
        }
    }
}

The method has two branches, one for lazy lists, and another one for non-lazy lists. Let us only focus on the second branch for now:

for @mods -> $mod {
    $mod
        ?? take $more mod $mod
        !! Failure.new(X::Numeric::DivideByZero.new:
                       using => 'polymod', numerator => $more);
    $more = $more div $mod;
}

take $more;

OK, the last take takes the remainder, that’s easy. In the loop, you divide the number by the next unit and then ‘count’ the intermediate reminder.

I would say I would implement it differently and switch the operators:

  for @mods -> $mod {
      $mod
-           ?? take $more mod $mod
+           ?? take $more div $div
          !! Failure.new(X::Numeric::DivideByZero.new:
                         using => 'polymod', numerator => $more);
-      $more = $more div $mod;
+      $more = $more mod $mod;
  }

  take $more;

With this code, I can get the number of hundreds, tens, and ones in 121:

> 121.polymod(100, 10, 1)
(1 2 1 0)

OK, let’s avoid two 1s:

> 1234.polymod(1000, 100, 10, 1)
(1 2 3 4 0)

Also works fine with the earlier example with seconds:

> 132768.polymod(86400, 3600, 60)
(1 12 52 48)

It is 1 day, 12 hours, 52 minutes, and 48 seconds.

As you see, now you have to use explicit units (8600 instead of 24) and you have to sort them in descending order, but now I can understand and explain the result, which I could hardly do for the original method.

 

57. Examining the Real role of Perl 6, part 1

During the last few days, we talked a lot about the Real role. Lets us then look at it more precisely. The code is located in the src/core/Real.pm file.

It contains the role itself and a few subroutines implementing different infixes. The Real role in its turn implements the Numeric role:

my role Real does Numeric {
    . . .
}

It is interesting that the class definition also needs some knowledge about the Complex class, that’s why there is a forward class declaration in the first line of the file:

my class Complex { ... }

The Real role defines many trigonometrical functions as methods, and as we already saw, they are using the Bridge method:

method sqrt() { self.Bridge.sqrt }
method rand() { self.Bridge.rand }
method sin() { self.Bridge.sin }
method asin() { self.Bridge.asin }
method cos() { self.Bridge.cos }
method acos() { self.Bridge.acos }
method tan() { self.Bridge.tan }
method atan() { self.Bridge.atan }

Another set of methods include generic methods that manipulate the value directly:

method abs() { self < 0 ?? -self !! self }
proto method round(|) {*}
multi method round(Real:D:) {
    (self + 1/2).floor; # Rat NYI here, so no .5
}
multi method round(Real:D: Real() $scale) {
    (self / $scale + 1/2).floor * $scale;
}
method truncate(Real:D:) {
    self == 0 ?? 0 !! self < 0 ?? self.ceiling !! self.floor
}

There’s a really interesting and useful variant of the round method, which allows you to align the number to the grid you need:

> 11.5.round(3)
12
> 10.1.round(3)
9

Another set of methods are used to convert a number to different data types:

method Rat(Real:D: Real $epsilon = 1.0e-6) { self.Bridge.Rat($epsilon) }
method Complex() { Complex.new(self.Num, 0e0) }
multi method Real(Real:D:) { self }
multi method Real(Real:U:) {
    self.Mu::Real; # issue a warning;
    self.new
}
method Bridge(Real:D:) { self.Num }
method Int(Real:D:) { self.Bridge.Int }
method Num(Real:D:) { self.Bridge.Num }
multi method Str(Real:D:) { self.Bridge.Str }

And here we have a problem in the matrix. The Bridge method is defined in such a way that it calls the Num method. In its turn, Num is calling Bridge, which calls Num.

Run one of the following lines of code, and Rakudo will hang:

Real.new.say;
Real.new.Str;