Posts tagged 'shell'

CLI command to whitelist Composer packages

published on December 04, 2017.

James asked this question the other day on Twitter:

#LazyWeb is there a way to do a composer update of everything except a specific package or two? like `composer update --exclude doctrine/orm --exclude doctrine/dbal` or something? I don't want to have to whitelist everything all the time (there's bigger problems ofc)

Given that Composer has no --exclude flag or similar, the only other option is to create a list of packages we allow to be updated, excluding the ones we don’t want to be updated. We need to create a whitelist.

Creating it manually would be a PITA though, especially if there’s a lot of packages to include or exclude.

CLI to the rescue!

composer info | grep -v ^doctrine | sed 's/  \+/:/g' | cut -d: -f1 | paste -sd\ 

Note: There’s a single whitespace after the last backslash \.

This would result in a list of packages in a single line, something like:

beberlei/assert composer/ca-bundle container-interop/container-interop guzzlehttp/guzzle mockery/mockery

Let’s break it down

The composer info command shows information about the installed packages. The output is in the format of:

vendor1/package1      vx.y.z      Package 1 description
vendor1/package2      vx.y.z      Package 2 description
vendor2/package       vx.y.z      Package description

It’s all text so we can work with that.

The next step is to remove the packages we don’t want to be in our whitelist. We do that with grep -v ^package1 — search for and output anything that does not start with package1.

We are only interested in the vendor/package parts of the composer info output as that’s all we’ll need eventually for the composer update command.

When we have text that is formatted in columns, we can use the cut command to split these columns by a delimiter. There is a delimiter in the above output from composer info, but the delimiter is a varying number of whitespaces. That’s not really helpful.

What can we do now? Using sed we can replace those whitespaces to something that’s easier to use as a delimiter in cut, a colon : for example. sed 's/ \+/:/g' searches for two or more consecutive spaces and replaces them with a single : (not really visible, but the / \+/ part has two space characters between / and \).

The output at this point would look something like this:

vendor1/package1:vx.y.z:Package 1 description
vendor1/package2:vx.y.z:Package 2 description
vendor2/package:vx.y.z:Package description

Now we can use the cut command, tell it to use the colon as a delimiter with -d: and to take only the first field with -f1.

Finally, we use the paste command to merge lines together to get the final output. The s option is to merge horizontally and the d\ tells it to join using a single space character (again, it’s not really visible, but there is a single space character after the \ character).

Feel free to convert this one liner to a shell script that takes the package names as arguments so it’s a bit more reusable for future uses :)

Happy hackin’!

Tags: bash, cli, composer, shell.
Categories: Development, Software.

Visual sudo for shell scripts

published on November 06, 2017.

The other day I was putting together a small shell script to do some administrative tasks on my Fedora workstation.

Even though I spend most of my time in a terminal, I wanted to have this script available from “everywhere”, that is to have it available to run it as a keyboard shortcut.

The script requires sudo privileges, and up until now, I thought that the only way to get sudo was from the terminal.

But then I remembered that some applications, like firewall-config, ask for the sudo password via a pop-up window. Surely it’s available to whatever application needs it, right?

The answer is pkexec.

To quote the man pages:

pkexec allows an authorized user to execute program as another user. If program is not specified, the default shell will be run. If username is not specified, then the program will be executed as the administrative super user, root.

Looks like that this pkexec is a part of, or at least relates to, something called Polkit. I honestly don’t understand that part yet, and what does it really do. Need to learn more about it, but as this is the first time I came across it, the learning more about it thing might not happen soon.

To make the shell script ask for the sudo password through pkexec we add it to the she-bang line:

#!/usr/bin/pkexec /bin/bash

touch /some/path/requiring/permissions.txt

Now when we run this script either through the terminal, or through an application finder/launcher applet, or by invoking it with a keyboard shortcut, it’ll ask for the sudo password first with the pop-up window.

Happy hackin’!

Tags: fedora, gksudo, pkexec, shell, sudo.
Categories: Development, Software.

Find duplicate lines from a CSV

published on March 21, 2016.

Finding duplicate lines from a CSV file is something I have to do from time to time, yet not on a regular enough basis to remember it all. Plus, I’m trying to blog more often.

cut -d, -f1 file.csv | tr -d '"' | sort | uniq -dc

cut to split the lines at the commas and select the first field, then tr to delete any double quotes that encloses the field, then sort and finally with uniq to show only the duplicated lines and to prefix every line with the count of occurrences.

Tags: linux, shell.
Categories: Development.

Notes on shell scripting

published on December 29, 2011.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

Yesterday I did some shell scripting and thought about writing down the few things learned along the way. Amazing how little needs to be done to learn a lot :)

Result of a command to a variable

First thing I learned is how to “save” the result of a shell command to a local variable:

PHP_BINPATH=$(which php)

By enclosing the command in parenthesis and putting a dollar sign in front of it, will put the result of that command in the variable.

An empty variable

Turns out, a variable can be empty, null. Nothing strange with that, until one tries to do something with that variable. For example:

PHP_BINPATH=
if [ $PHP_BINPATH == "foo" ]
  then
    echo "It's foo"
fi

will die with a strange error: “line 2: [: =: unary operator expected”. Problem is that when evaluating it will see if [ == “foo” ] and turns out [ is some reserved command or some such. The fix is to wrap $PHP_BINPATH in double quotes:

PHP_BINPATH=
if [ "$PHP_BINPATH" == "foo" ]
  then
    echo "It's foo"
fi

Pass all the arguments!

When calling some other command from within the shell script, and all the arguments which are passed to the shell script need to be passed to that other command, use ”$@” for that:

$PHP_BINPATH usher.php "$@"

This will pass all the arguments to the PHP script which is called from within the shell script.

Happy hackin’!

Change permissions on folders/files only

published on January 11, 2011.
Heads-up! You're reading an old post and the information in it is quite probably outdated.

This is just a quick reminder for myself. Should really remember this one. Changes permissions on folders|files only.

$ find /path/to -type d -exec chmod 775 {} \;
$ find /path/to -type f -exec chmod 664 {} \;
Tags: permissions, shell.
Categories: Development.
Robert Basic

Robert Basic

Software engineer, consultant, open source contributor.

Let's work together!

If you require outsourcing or consulting help on your projects, I'm available!

Robert Basic © 2008 — 2018
Get the feed