PHP FPM slow log

published on November 23, 2017.

The other day I was going through the configuration file for php-fpm, when I noticed a configuration directive I haven’t before: slowlog. I guess it’s been around for a while, I just never noticed it.

The php-fpm slow log is a pool configuration, meaning that we configure it in www.conf, and has two directives for it:

  • the slowlog, which is a path to a file where the slow requests will be logged,
  • and request_slowlog_timeout is a time unit after which PHP will dump a backtrace for that request in to the slow log file. We can configure it to be in seconds, minutes, hours, or days.

What’s in the box backtrace?

It has the date and time for when the slow request happened, the pool and PID for the php-fpm process. script_filename is the entry point to the request, and the backtrace includes a list of function calls up until the moment when the request_slowlog_timeout was hit.

[23-Nov-2017 15:28:21]  [pool www] pid 8992
script_filename = /var/www/example/web/app_dev.php
[0x00007efe32a14a40] sleep() /var/www/example/src/AppBundle/Controller/DefaultController.php:18
[0x00007efe32a149d0] indexAction() /var/www/example/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpKernel.php:153
[0x00007efe32a14960] call_user_func_array() /var/www/example/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpKernel.php:153
[0x00007efe32a14470] handleRaw() /var/www/example/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/HttpKernel.php:68
[0x00007efe32a14320] handle() /var/www/example/vendor/symfony/symfony/src/Symfony/Component/HttpKernel/Kernel.php:169
[0x00007efe32a14250] handle() /var/www/example/web/app_dev.php:29

Even though it doesn’t reveal too much, together with other profiling tools, like Xdebug and kcachegrind, it can help us a great deal on finding and fixing performance problems in web applications.

Happy hackin’!

Visual sudo for shell scripts

published on November 06, 2017.

The other day I was putting together a small shell script to do some administrative tasks on my Fedora workstation.

Even though I spend most of my time in a terminal, I wanted to have this script available from “everywhere”, that is to have it available to run it as a keyboard shortcut.

The script requires sudo privileges, and up until now, I thought that the only way to get sudo was from the terminal.

But then I remembered that some applications, like firewall-config, ask for the sudo password via a pop-up window. Surely it’s available to whatever application needs it, right?

The answer is pkexec.

To quote the man pages:

pkexec allows an authorized user to execute program as another user. If program is not specified, the default shell will be run. If username is not specified, then the program will be executed as the administrative super user, root.

Looks like that this pkexec is a part of, or at least relates to, something called Polkit. I honestly don’t understand that part yet, and what does it really do. Need to learn more about it, but as this is the first time I came across it, the learning more about it thing might not happen soon.

To make the shell script ask for the sudo password through pkexec we add it to the she-bang line:

#!/usr/bin/pkexec /bin/bash

touch /some/path/requiring/permissions.txt

Now when we run this script either through the terminal, or through an application finder/launcher applet, or by invoking it with a keyboard shortcut, it’ll ask for the sudo password first with the pop-up window.

Happy hackin’!

Tags: sudo, shell, pkexec, gksudo, fedora.
Categories: Development, Software.

What implements an interface

published on November 02, 2017.

Creating and implementing interfaces in our code is important. It helps with swapping out components, eases testing, separates the what from the how.

But, it’s not enough just to slap an interface on a class and be done with it.

We also need to consider on what are we putting that interface on.

An example

Say, we’re creating a queuing system for an RSS feed reader. We can tell the queue to queue the feed URLs. Depending on our needs, we can use something like RabbitMq, or a database, to use as a queuing mechanism.

We haven’t decided on that yet, but either way, we start with an interface for this imaginary queue:

<?php declare(strict_types=1);

namespace Example\Infrastructure\Queue;

use Example\Domain\Rss\FeedUrl;

interface FeedUrlQueue
{
    public function add(FeedUrl $feedUrl);
}

By having this nice little interface, we can TDD the part of the code that will use an implementation of this interface.

After a while we decide we’ll go with a database queuing mechanism first, so we create an implementation for the FeedUrlQueue interface:

<?php declare(strict_types=1);

namespace Example\Infrastructure\Storage\Database;

use Example\Domain\Rss\FeedUrl;

class FeedUrlTable extends AbstractTable implements FeedUrlQueue
{
    public function add(FeedUrl $feedUrl)
    {
        $qb = $this->getQueryBuilder();

        $query = $qb->insert('feed_urls')
            ->values(
                [
                    'url' => '?',
                ]
            )
            ->setParameter(0, (string) $feedUrl);

        $query->execute();
    }
}

That’s nice! We have an interface, a concrete implementation, and the possibility to write new implementations and swap them out with existing ones with little effort.

Job well done.

Is it done, let alone well?

Sure it is, I repeat, we have an interface, a concrete implementation, and the possibility to write new implementations and swap them out with existing ones with little effort.

Something’s fishy

There’s three things that stand out for me here, telling me that something is not quite right with this code.

First, a class that represents a Table, also is a FeedUrlQueue. It really shouldn’t be two things at the same time. It either should be a queue, or a table, most certainly not both.

Second, a class whose only responsibility should be to store an URL into a database, no matter from where that URL comes from, is now limited to store feed URLs that come from the queue. OK, this may, or may not be, a legitimate limitation we decided on.

And third, it is also responsible to figure out how can it transform a FeedUrl domain object into a string that can be stored in the database. Does it have a __toString magic method, so we can cast it to a string? Or maybe it’s legacy code so it has one of those toString() method which we need to call? We don’t know without looking.

Killing three giants with one stone

A better, a correct way, would be to have something like a DatabaseFeedUrlQueue that implements the FeedUrlQueue, and uses the FeedUrlTable:

<?php declare(strict_types=1);

namespace Example\Infrastructure\Queue;

use Example\Domain\Rss\FeedUrl;

class DatabaseFeedUrlQueue implements FeedUrlQueue
{
    protected $table;

    public function __construct(FeedUrlTable $table)
    {
        $this->table = $table;
    }

    public function add(FeedUrl $feedUrl)
    {
        $payload = [
            'url' => (string) $feedUrl
        ];
        $this->table->save($payload);
    }
}

and the FeedUrlTable becomes something like this:

<?php declare(strict_types=1);

namespace Example\Infrastructure\Storage\Database;

class FeedUrlTable extends AbstractTable
{
    public function save(array $payload)
    {
        $qb = $this->getQueryBuilder();

        $query = $qb->insert('feed_urls')
            ->values(
                [
                    'url' => '?',
                ]
            )
            ->setParameter(0, $payload['url']);

        $query->execute();
    }
}

By refactoring the code like this, we pretty much fix all three problems at once:

  • a DatabaseFeedUrlQueue is a FeedUrlQueue, and the FeedUrlTable can stop being two things at once;
  • there’s a clearer separation of concerns, the DatabaseFeedUrlQueue is responsible to create the payload, and FeedUrlTable is responsible to store it;
  • the storage layer knows nothing about our domain objects and how to use them.

Yes, now we have one more class to maintain, but the overall maintainability, I believe, is reduced, as it is much clearer what each class does.

Happy hackin’!

Smarter tag search in Vim

published on November 01, 2017.

As part of my Vim setup for PHP development, I use the vim-php-namespace plugin to add use statements in my PHP code.

vim-php-namespace uses the tags file to find the class and the namespace it belongs to, and then adds it to the rest of the use statements.

It all works great, but there are times when it shows too much possibilities.

For example, when I want to import the namespace for the Transaction class, it finds the correct Transaction class, but it also finds functions called transaction in my codebase, and then gives me a choice what I want to import:

See? One class (kind c), and two functions (kind f).

I could exclude functions from being generated in tag files, but that’s not really an option because there are times when I need the functions tags.

I dove into the vim-php-namespace source code, determined to get rid of this “functionality”.

Turns out the plugin actually uses a Vim command, called ptjump, to search the tags file and show the preview window, so the user can pick out the correct tag in case there’s more than one.

Of course there’s an option for that

Then I started reading the help pages for tags in more detail, and after a while I found the answer: tagcase.

To quote the help file:

This option specifies how case is handled when searching the tags file.

And it has the following options:

  • followic Follow the ‘ignorecase’ option
  • followscs Follow the ‘smartcase’ and ‘ignorecase’ options
  • ignore Ignore case
  • match Match case
  • smart Ignore case unless an upper case letter is used

I’ve set it to smart and, well, now it does what I want it to do:

set tagcase=smart

It correctly finds only one match for the Transaction class and the plugin inserts the use statement for it. Yey!

Happy hackin’!

Tags: php, vim, tags, namespace, plugin.
Categories: Development, Software.

Reading from standard input with Go

published on October 30, 2017.

Last year I started learning Go, then I stopped, and now I don’t remember anything what I learned.

Going to try it a bit different this time, by writing down what I do, learn, experiment.

As I don’t have a specific thing I want to build with Go, I’m just going to do simple scripts and examples, to get to know Go’s language specification and built-ins as much as possible.

Hopefully, there won’t be too much mistakes in my code samples. I’ve set up a golearn repository just for this, so feel free to create issues/PRs telling me where I went wrong.

Setting up the Go environment

I installed Go to the location that the official docs recommend — /usr/local, and I created the “Go workspace” in ~/projects/gocode and then I put my code under ~/projects/gocode/src/github.com/robertbasic/golearn.

I do find it kinda weird that everything lives in one “workspace”. But, whatever.

As for the editor, I’m using Visual Studio Code with the Go plugin.

I first tried using the vim-go plugin, but I couldn’t make it work in a custom runtimepath. I didn’t want to brick the Vim setup I have for my day-to-day work, so that’s why I tried a custom location.

Next I tried the Gogland from JetBrains, but when I ran it, my laptop sounded like a jet when the fans kicked off, so… yeah.

Then I tried Visual Studio Code, installed the Go plugin and I was pleasantly surprised when the editor figured out I was missing some Go programs like gofmt, golint, etc., and offered me to install them all. Great user experience for newcomers to Go.

About those docs…

After printing out the required “Hello, world!” to the terminal, I set out to figure out how to read in stuff from the standard input.

I would really love to say I figured it out all on my own how to do it, just by reading the official documentation, but I couldn’t. Maybe I don’t know how to use Go’s docs, maybe I’m spoiled by PHP’s excellent documentation, but after an hour or so flipping through the docs, I ended up Googling the solution.

From the documentation, I couldn’t figure out does the solution lie within the io, or the bufio package. Maybe both? I’ll have to tinker with them some more to understand the difference between the two.

From all the answers I found on the internet about this, I went with what seems the simplest way of reading from standard input: using a bufio.Scanner. Again, I’ll have to poke more around this, to see all the different ways I can accomplish this and compare the solutions.

The first “program”

The goal is to read whatever the user types in to standard input, line by line.

First, I have a slice of strings. Not an array, but a slice. I believe the main difference is that an array has a set size, while a slice’s size can change during execution.

// Inputs holds the inputs read in from stdin
type Inputs []string

I’m not yet 100% sure does this create a new type “Inputs” which is composed of a slice of strings, or is it just a slice of strings named “Inputs”? I think it’s the former, because I can use it as a return type-hint in a function.

Then in the main() I get a new scanner for the standard input:

scanner := bufio.NewScanner(os.Stdin)

It returns a pointer to the Scanner, so I pass it as such to my ReadInput function:

is := ReadInput(*scanner)

The ReadInput function takes a scanner of bufio.Scanner type and returns Inputs:

func ReadInput(scanner bufio.Scanner) Inputs

I declare two variables, one to hold the string read in from stdin, the other to “collect” all the input strings:

	var t string
	var i Inputs

Then I create an endless for loop, from which I break out when the user enters a new line only. In this loop the Scanner scans the stdin, and gets the text from it. If the text entered is not empty, I append it to the Inputs, as it is a slice of strings:

	for true {
		fmt.Print("Add a new todo [empty to stop reading]: ")
		scanner.Scan()
		t = scanner.Text()

		if err := scanner.Err(); err != nil {
			fmt.Println("Error reading from input: ", err)
		}

		// scanner.Text() strips new lines
		// so in case of just a new line
		// it's actually an empty string
		if t == "" {
			break
		}

		i = append(i, t)
	}

and finally I return the Inputs from the ReadInput function:

    return i

Back in main I do a “sanity” check by printing out the number of lines read in, and printing them out one by one:

	fmt.Printf("Got %d inputs from stdin:\n", len(is))

	for _, v := range is {
		fmt.Println(v)
	}

When executed, it looks something like this:

$ go run readstdin.go 
Hello, world!
Add a new todo [empty to stop reading]: Learn go
Add a new todo [empty to stop reading]: Write blog posts
Add a new todo [empty to stop reading]: 
Got 2 inputs from stdin:
Learn go
Write blog posts

The entire file is available here.

With this I learned a bit about Go’s types, creating functions, using the for loop, pointers, slices vs. arrays, and, of course, reading stuff from the standard input.

Happy hackin’!

P.S.: It feels good to have a beginners mind, again, about something.

Robert Basic

Robert Basic

Software engineer, consultant, open source contributor.

Let's work together!

If you require outsourcing or consulting help on your projects, I'm available!

Robert Basic © 2008 — 2019
Get the feed