Image for post
Image for post

Ops Scripting with Perl: Frequency

Automation with scripting languages is required core skills for operations oriented roles. Ages ago, when Unix (later Linux) scripting involved shell programming and tools like , , , , , etc. and later Perl became the popular go-to language for systems administrators.

Perl is still popular today for systems automation and operating systems configuration. If cannot be removed from a Linux distro, for example, as many underlying automation depends on Perl to function. For these examples, Perl still outperforms other languages: 1100% faster than Python, 800% than faster than ruby.

The Problem

The problem is to print a formatted summary of the shell and number of users that use that shell. We’ll have a local file to as the data input. In Perl, we’ll create a frequency hash to store our counts.

The Data

Here’s the local file used for this exercise:

The Output

The output given the report given the data from above would have these counts:

The Starting Code

Here’s some initial starting code to get you started, which, given a hash of , prints out a formatted report.

Output Report with %count hash

For those that are familiar with the C-Language , this should not be unfamiliar. Perl also has a repetition operator to repeat several characters.

What might be a surprise for those unfamiliar with Perl is to express a loop in a single line (line 10), which can be unfolded into this below:

Other language features include using as a default index, when one is not defined, and using a space instead of parenthesis to pass parameters. The same above, could be rewritten as this:

The Solutions

Unlike the previous articles I wrote for Ruby and Python, I decided to put this in a single article. I assume most of the audience is looking at this article out of curiosity to see how Perl compares to either Ruby or Python.

Solution 1: Slicing Shell First

Here’s a common method to open a file, check for an error, and then use the spaceship operator to return a list to a conditional loop.

Each iteration of the loop will have the line available as the variable .

Slice out the Shell

Each cycle of this loop, we off the newline character, the line into a list, and then slice off the 7th item (indexed by 6), which represents the shell.

With this single line (line 8):

We do the following:

  1. Determine if we have actually have a shell (not undefined value)
  2. Reference the hash by the shell, returning a if it is not set in a previously.
  3. Increment its value by

Perl is kind enough to use an intelligent default of if we have not initialized the value. This saves unnecessary branch logic to check if key exists before, and initialize it to a default.

Solution 2: Regex Filter

This is similar to the problem before, except that we check to see if we have a valid string first using a regex (regular expression) to filter out invalid lines. The default scratch variable is used here to represent the line.

Regex to filter out invalid lines

Because we filter out invalid lines that do not have a shell specified, we can just slice out the shell key and both create the key-value pair as well as increment it in one line, essentially doing at least four operations in one line:

This would be the same as this:

Solution 2: Functional Style

Instead of using a conditional loop with this solution, we use a pipeline to feed into and .

Grep and Map

This will run the following pipeline logic:

Alright, so this is something truly magical, and may need some explaining. We’ll dissect each link of this pipeline, as if it was written like this:

The first part of the link, reading from back to front, is the filtering out bad lines (that is ones that do not have a 7th column and end with a colon):

The next pipeline link transforms the incoming items to just the shell:

For the Perl , it is using the default fed into this code block, and thus could be rewritten as this to be explicit:

In the last link, we output a hash by specifying a key-value pair, instead of a regular element.

Perl will see that you are using key-value pair and know this map is producing a hash, and not a list. In other languages, you would need to explicitly coerce it to a hash (or dictionary).

Thus we are essentially doing this:

Here are some examples of using this method:

So in our example, we produce pairs and send them to our hash:

But wait, there’s something else interesting happening here, see if you can catch it?

Here, map is dynamically building a hash as it is processing, which overwrites the hash.

The Conclusion

I wanted to give people a sense of the richness of Perl and show how it may compare to Ruby and Python for similar problems, as well as offer an introduction to Perl.

These are some general takeaways for the language and this tutorial:

  • Variables in Perl are , , or and will default to or empty string (in case that was not obvious).
  • A scratch scalar variable is used in looping constructs and is the default variable input for many functions or operators in Perl.
  • Formatted output with and repetition operator
  • Opening a file with and handling errors with
  • Spaceship operator to generate lines from a file handle
  • Splitting strings into lists with
  • Slicing off list elements
  • regex match operators with and
  • filtering elements with
  • transforming elements with
  • building hashes with

Written by

Linux NinjaPants Automation Engineering Mutant — exploring DevOps, Kubernetes, CNI, IAC

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store