What is xargs?

xargs is one of my favourite cli tools that I use almost daily. There's not a lot to it, but I'd like to show you some of the things you may be able to do with it. In this little post, I shall be using the GNU version of xargs. You may be using a slightly different version, but for the most part things should be more or less the same.

xargs allows you to build and then run commands based on data you can pass through via stdin. Let's start with a small example:

echo -n "1\n2\n3" | xargs -n 1

If we were to echo "1\n2\n3" normally, we'd just expect:

1
2
3

as our output. By default, xargs will separate these per each line. In this case we print a new line after each number (\n being a new line character).

We could change that delimiter to another character if we wish. By passing -d along with our character of choice, in this case our new delimiter is a, as you can see by our original string, 1a2a3.

echo -n "1a2a3" | xargs -n 1 -d a

Note that we don't pass a at the end of the sequence as 3 is our last character. So what is this -n business all about?

In the case of echo -n, we're just stating that we do not wish to print a new line after our string. And in the case of xargs -n 1, we are passing through the --max-args. This means we can run a command per each argument.

You'll notice that if we were to run:

echo -n "1a2a3a4" | xargs -n 2 -d a

our output will be split between 2 lines instead of 4. So what can we do now that our output it being printed onto new lines, split by a delimiter?

We could be boring and just echo out each argument (which happens by default anyway):

echo -n "1\n2\n3" | xargs -n 1 echo

So when you write the above, what's happening is something like the following:

for [1, 2, 3] as $arg
    echo $arg
end

which means that our argument (per line) is passed to the command we specify at the end. In this case, it's echo.

Let's move on to something a little more interesting. We can list Docker containers using the docker ps command. If we run:

docker ps -q

we'll see that we just get our container IDs. Each on a new line. In comes xargs! Let's kill those containers:

docker ps -q | xargs -n 1 docker kill

It's that simple! So again, what's happening here is something a bit like:

for $containerIDs as $arg
    docker kill $arg
end

But what if we want to run a more complex command that doesn't simply just end with the xargs argument? That's simple too! Let's try using the cat program instead of echo this time. First we'll create a new file though:

echo "Harry\nPedro\nWillis\nSimon\nJimmy\nSammy\nLester\nAlice" > names

So in our new file, named names, we should have:

Harry
Pedro
Willis
Simon
Jimmy
Sammy
Lester
Alice

Let's cat that file into xargs:

cat names | xargs -n 1

Now let's echo out each name as before:

cat names | xargs -n 1 echo

And now let's add some text before and after each name:

cat names | xargs -n 1 -I R echo "I am" R"."

I'll break this down a little bit. We are using -I here, which acts as a replacement variable for our argument. In this case it's been defined as R. It doesn't have to be R, for example it could be NAME:

cat names | xargs -n 1 -I NAME echo "I am" NAME"."

So we can define it using -I NAME and then use NAME later on in our command, wrapped by some text before and after. In this case the text before is just I am and the text after is just a period.

So what else can we do? Lots, actually! I'm going to show you one more cool little trick with xargs. And that is to run commands concurrently by using the --max-procs option. We can define how many processes will run at the same time with this option, or just -P for shorthand. Let's do that:

cat names | xargs -n 1 --max-procs 2 -I R echo "I am" R"."

So here we are saying that we want to run at most, 2 processes at the same time. If we pass 0 here instead:

cat names | xargs -n 1 --max-procs 0 -I R echo "I am" R"."

it will use the max number of processes it can. With such a small amount of data that we're using, we shouldn't see any differences in the output. If you were to run a command that may take a little longer, such as the docker kill command from earlier, you might see the containers being killed in a seemingly random order:

docker ps -q | xargs -n 1 --max-procs 0 docker kill

Give it a try yourself! And remember, man is always your friend. There are lots of other options that I have not covered here in this post. I suggest you have a look for yourself and see what interesting things you can get done with the xargs program.