What does "xargs grep" do?

Question

I know the grep command and I am learning about the functionalities of xargs, so I read through this page which gives some examples on how to use the xargs command.

I am confused by the last example, example 10. It says "The xargs command executes the grep command to find all the files (among the files provided by find command) that contained a string ‘stdlib.h’"

$ find . -name '*.c' | xargs grep 'stdlib.h'
./tgsthreads.c:#include
./valgrind.c:#include
./direntry.c:#include
./xvirus.c:#include
./temp.c:#include
...
...
...

However, what is the difference to simply using

$ find . -name '*.c' | grep 'stdlib.h'

?

Obviously, I am still struggling with what exactly xargs is doing, so any help is appreciated!

This question may be helpful: When is XArgs needed?
– TheOdd
Oct 4, 2016 at 21:56 — Oct 4, 2016 at 21:56

score 45 · Accepted Answer · 2018-04-04 09:57:51Z

$ find . -name '*.c' | grep 'stdlib.h'

This pipes the output (stdout)* from find to (stdin of)* grep 'stdlib.h' as text (ie the filenames are treated as text). grep does its usual thing and finds the matching lines in this text (any file names which themselves contain the pattern). The contents of the files are never read.

$ find . -name '*.c' | xargs grep 'stdlib.h'

This constructs a command grep 'stdlib.h' to which each result from find is an argument - so this will look for matches inside each file found by find (xargs can be thought of as turning its stdin into arguments to the given commands)*

Use -type f in your find command, or you will get errors from grep for matching directories. Also, if the filenames have spaces, xargs will screw up badly, so use the null separator by adding -print0 and xargs -0 for more reliable results:

find . -type f -name '*.c' -print0 | xargs -0 grep 'stdlib.h'

*added these extra explanatory points as suggested in comment by @cat

you might consider noting (because it seems you omitted) the key point that | pipes stdout to grep's stdin which is not the same as grep's arguments and gives confusing results. — Oct 5, 2016 at 0:28
Or use GNU find's find -name '*.c' -exec grep stdlib.h {} +. I pretty much never actually use xargs. Also surprised nobody mentioned that xargs serves a similar purpose to grep $(find) command substitution, so I wrote an answer of my own. Explaining xargs as command substitution with fewer limitations and problems seems natural. — Oct 5, 2016 at 2:18
One situation I use xargs on is if I am deleting a lot of files as a result of find. If you just do -exec rm, it will run rm on each file one at a time, which is very inefficient. Piping to xargs will do them all at once with one rm. Limiting with say -n50 (do 50 at a time) could prevent command line overflow (problem with a lot of files). — Oct 5, 2016 at 12:47
@lsd: Why not find -delete for that special case? Or for commands other than rm, if you have GNU find, then -exec some_command {} + groups into batches like xargs, instead the \; behaviour of running the command separate for each. — May 23, 2017 at 20:15
@lsd find runs command on each file if and only if it's using -exec command \; Both xargs and -exec command \+ will call the command with maximum number of arguments allowed by the system. In other words, they're equivalent — May 23, 2017 at 22:21

score 8 · Answer 2 · 2017-05-23 22:43:47Z

xargs takes its standard input and turns it into command line args.

find . -name '*.c' | xargs grep 'stdlib.h' is very similar to

grep 'stdlib.h' $(find . -name '*.c')  # UNSAFE, DON'T USE

And will give the same results as long as the list of filenames isn't too long for a single command line. (Linux supports megabytes of text on a single command line, so usually you don't need xargs.)

But both of these suck, because they break if your filenames contain spaces. Instead, find -print0 | xargs -0 works, but so does

find . -name '*.c' -exec grep 'stdlib.h' {} +

That never pipes the filenames anywhere: find batches them up into a big command line and runs grep directly.

\; instead of + runs grep separately for each file, which is much slower. Don't do that. But + is a GNU extension, so you need xargs to do this efficiently if you can't assume GNU find.

If you leave out xargs, find | grep does its pattern matching against the list of filenames that find prints.

So at that point, you might as well just do find -name stdlib.h. Of course, with -name '*.c' -name stdlib.h, you won't get any output because those patterns can't both match, and find's default behaviour is to AND the rules together.

Substitute less at any point in the process to see what output any part of the pipeline produces.

Further reading: http://mywiki.wooledge.org/BashFAQ has some great stuff.

GNU xargs also has -d to set the separator, so you can use -d'\n' to handle a newline-separated list, which may be useful if you handle a list of file names in a file, etc. (as long as the file names don't have newlines in them, that is.) — Oct 5, 2016 at 10:58
@ilkkachu: yeah, newlines in filenames are a lot more rare than spaces, since they break most scripts. myfunc(){ local IFS=$'\n'; fgrep stdlib.h` $(find); } also works with the same effect. Or as a one-liner, a (IFS=...; cmd...) subshell also works to contain the change to IFS without having to save/restore it. — Oct 5, 2016 at 12:35
@PeterCordes Please don't do command $( find ) type of stuff. Problematic filenames with spaces and special characters can break this type of thing. At the very least double quote the command substitution. — May 23, 2017 at 22:31
@SergiyKolodyazhnyy: Thanks for pointing out that it looks like I'm actually recommending doing that. People skimming might have copy/pasted that instead of reading the next section. Updated to address that. — May 23, 2017 at 22:45
@SergiyKolodyazhnyy: Or were you replying to my comment? Notice that I set IFS so it's equivalent to using xargs '-d\n'. Glob expansion and shell metacharacter processing happens before command substitution's effects, so I think it's safe even with filenames that contain $() or >. Agreed that using word-splitting on command substitution is not good practice except for one-off interactive use where you know something about the filenames. But command "$(find)" is only useful if you expect it to produce exactly 1 filename... — May 23, 2017 at 22:53

Luis Alvarado 212k168168 gold badges547547 silver badges713713 bronze badges · Answer 3 · 2016-10-04 22:21:37Z

In general, xargs is used for cases where you would pipe (with the symbol |) something from one command to the other (Command1 | Command2), but the output from the first command is not correctly received as the input for the second command.

This typically happens when the second command does not handle data input through Standard In (stdin) correctly (eg: Multiple lines as input, the way the lines are setup, the characters used as input, multiple parameters as input, the data type received as input, etc..). To give you a quick example, test the following:

Example 1:

ls | echo - This will not do anything since echo does not know how to handle the input he is receiving. Now in this case if we use xargs it will process the input in a way that can be handled correctly by echo (eg: As a single line of information)

ls | xargs echo - This will output all the information from ls in a single line

Example 2:

Let's say I have multiple goLang files inside a folder called go. I would look for them with something like this:

find go -name *.go -type f | echo - But if the pipe symbol there and the echo at the end, it would not work.

find go -name *.go -type f | xargs echo - Here it would work thanks to xargs but if I wanted each response from the find command in a single line, I would do the following:

find go -name *.go -type f | xargs -0 echo - In this case, the same output from find would be shown by echo.

Commands like cp, echo, rm, less and others that need a better way to handle the input get a benefit when used with xargs.

score 4 · Answer 4

xargs is used to auto generate command line arguments based (usually) on a list of files.

So considering some alternatives to using the followoing xargs command:

find . -name '*.c' -print0 | xargs -0 grep 'stdlib.h'

There are several reasons to use it instead of other options that weren't originally mentioned in other answers:

find . -name '*.c' -exec grep 'stdlib.h' {}\; will generate one grep process for every file—this is generally considered bad practice, and may put a big load on the system if there are many files found.
If there are a lot of files, a grep 'stdlib.h' $(find . -name '*.c') command will likely fail, because the output of the $(...) operation will exceed the maximum command line length of the shell

As mentioned in other answers, the reason for using the -print0 argument to find in this scenario and the -0 argument to xargs, is so that filenames with certain characters (e.g. quotes, spaces or even newlines) are still handled correctly.

What does "xargs grep" do?

What does "xargs grep" do?

4 Answers

相關文章